Method, system and user interface for creating and displaying of presentations

ABSTRACT

A method and system for creating at least one presentation, comprising: at least one computer; at least one application operable on the at least one computer, the at least one application configured for: receiving a plurality of assets; and allowing at least one user to define the at least one presentation over the time that the at least one presentation is to be displayed, using at least two assets to be simultaneously presented for at least one selected point in time.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.61/354,984, filed Jun. 15, 2010, which is incorporated by reference inits entirety.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1-5 and 15-16 arc figures describing various systems and methodsof multiple embodiments of the invention.

FIGS. 6-14 and 17-7 illustrate various screen shots and examples relatedto multiple embodiments of the invention.

DESCRIPTION OF SEVERAL EMBODIMENTS

FIG. 1 illustrates a system 100 for creating (e.g.. defining), playingback and otherwise interacting with information presentations comprisinginteractive and non-interactive information, according to oneembodiment. The system 100 allows at least one person to: define and/oredit and/or interact with at least one information presentation (e.g.,either before a user views the information presentation (sometimesreferred to as “predefined” or “on-demand”) and/or at the same lime auser views the information presentation (sometimes referred to as“real-time”)); view the defined information presentation; update and/orcontrol the information presentation (e.g., either before the user viewsthe information presentation and/or at the same time a user views theinformation presentation); interact with the information presentation:edit the information presentation: interact in real time (e.g.,synchronously, live) with other users of the system and users of theinformation presentation; or interact in non-real time asynchronously)with other users of the system mad users of the inform anonpresentation; or any combination thereof, hereinto referred to as the“capabilities of the system”. An information presentation can be definedin system 100 by utilizing a plurality of assets. The plurality ofassets can comprise non-interactive assets or interactive assets. Thenon-interactive assets 305 (see FIG. 3) (which can be assets that arenot interactive with either another person, system 100, or anothersystem) can comprise teal time assets 315 and, or non real-lime assets320. The real-time assets 315 can comprise: real time informationstreams, real-time audio steams, or real-time video streams, or anycombination thereof. The non real-time assets 320 can comprise: at leastone time based file (e.g., audio file, video file, multimedia file), orat least one document file (e.g., Microsoft Office Document, Worddocument, PDF, Excel, Visio, Flash, slide-show presentation), at leastone image file (e.g., picture), or any combination thereof. Theinteractive assets 310 (which are assets that can be interactive withanother person (e.g., user), system 100, or another system) can comprisewidgets.

The widgets can comprise at least one text chat widget, at least oneaudio/video chat widget, at least one asset browser widget, at least onediscussion widget, at least one audio recording/playback widget. atleast one video recording/playback widget, at least one streamrecording/playback widget, at least one audio/video editing widget, atleast one document viewing widget, at least one document editing widget,at least one linking widget, at least one activity widget, at least oneslide show editing/viewing widget, at least one diagram editing/viewingwidget, at least one drawing suite widget, at least one whiteboardwidget, at least one polling widget, at least one survey widget, atleast one assessment widget, at least one spreadsheet widget, at leastone user list widget, at least one source code viewing and editingwidget, at least one tracking widget, at least one, glossary widget, atleast one screen sharing and/or screen recording widget, at least onewiki widget, at least one bookmark widget, at least one calculatorwidget, at least one presentation index widget, at least one layerswidget, at least one integrations widget, or at least one third partyplatform interactive widget, or any combination thereof. It should benoted that any widget can be provided using third-party software. Inaddition, those of ordinary skill will see that any widget function(s)can be combined with any other widget function(s) to form other widgets.

Referring to FIG. 1, in one embodiment, server (e.g., host) computer 110can communicate over a network 105 (e.g., Intranet, Internet) withclient computer 115. A creation/playback application 120 can be accessedby a user utilizing the client computer 115 and/or the server computer110. The creation/playback application 120 can support a number of modeswhich include, but are not limited to: editor mode, run-time/playbackmode, record mode, or collaborate mode, or any combination thereof.Those of ordinary skill in the art will see that many other modes can becreated and implemented by the system 100 in the creation/playbackapplication 120. The system 100 can be set to be run in multiple modessimultaneously and/or use capabilities of the multiple modessimultaneously. The creation/playback application 120 can also: useeditor mode and editor mode capabilities to create and/or edit aninformation presentation; use runtime mode (e.g., playback mode) andrun-time mode capabilities to playback an information presentation; usecollaborate mode and collaborate mode capabilities to supportcollaboration within the application and system (above and beyond theinherent real-time capability of any individual asset or widget, such aschat or video chat); or use record mode to record any and all statechanges to an information presentation wherein such state changes may beplayed back or further edited; or any combination thereof. Thecapabilities of the modes ate further described below, as well asillustrated together with the embodiments of the creation/playbackapplication 120 executing some of the capabilities of the system 100.

In one embodiment, the creation/playback application 120, can run on theserver computer 110 (as shown in FIG. 1). In alternate embodiments, thecreation/playback application 120 can run on the client computer 115 ora combination of the server computer 110 and client computer 115. Thus,any of the modules in the creation/playback application 120 can resideon the server computer 110 or the client computer 115. For example, inone embodiment, some of the modules could reside on the server computer110 and some on the client computer 115. In other embodiments, all ofthe modules could reside on the server computer 110 or the clientcomputer 115.

Creation/playback Application

FIGS. 2A and 2B set forth details of the creation/playback application120, according to one embodiment. In one embodiment, thecreation/playback application 120 can receive information presentationdata. Information presentation data can include a plurality of assets,assets meta-data, other meta-data, or state change/time definition data,or any combination thereof.

At this point, it should be noted that state change/time definition datacan describe and/or define a change to the state of the informationpresentation (including describing changes in state for the creationplayback application itself and describing changes to other statechange/time definition objects/data) for a given point in time. Thestate change/time definition data can specify a state change to be:immediately executed, executed at some point in the future, or executedat some point in the past: or any combination thereof. In this way,state change/time definition data can collectively and together definean information presentation and an information presentation over time,and together with the creation/playback application 120, flexiblyaccommodate a number of capabilities of the system 100.

In one example of capabilities, a collection of -state change/timedefinition data for various points in the past and future can representan information presentation over time and can be received by thecreation/playback application. As previously mentioned, statechange/time definition objects, in addition to defining state changes toboth the application and to other state change/time definition objects,can define state changes to the information presentation, andspecifically, state changes as they pertain to the use or state ofassets, wherein assets, as well as assets meta-data is also received/tobe received by the state-time application. Using such informationpresentation data as described, the creation/playback application 120can: render the information presentation over time, visually on atimeline area and/or playback the information presentation by executingthe state change/time definition data at the appropriate times, thereby“playing back” the information presentation (e.g., by actuating,activating, and displaying the assets).

As previously detailed, state change/time definition data can be of animmediate run nature; i.e., defined to be executed immediately. In theprevious example, one use of immediate run state change/time definitiondata could be for initializing the application and informationpresentation. The capability of implementing immediate run slatechange/time definition data can support a number of other capabilitiesof the system. For example, immediate run state changes can be used toinitialize the application and the information presentation into a knownstate, and can allow for a user, at any point in time, including at theapplication's initialization, to access the information presentation atany point of the information presentation's potential duration, as wellas to sync the stale of the application that may be defined for thatpoint in time. Another example using this capability could be where asecond user can enter into the application and immediately be synced tothe state of the application and information presentation of a firstuser, thereby immediately replicating exactly what the first user seesat that point in time.

The creation/playback application 120 can continually listen for,receive, process, store and execute all information presentation data,including immediate-run state change/time definition data. In this way,the state of the creation/playback application 120 and the informationpresentation (and all of both of their constituent parts) can becontinually updated and modified in real-time. By providing foressentially any real time change and any real time update, any real-timecapability can be achieved, including that of various real-timecollaborative capabilities, including, but not limited to: real-timecapabilities within widgets (for instance real-time updates to the textchat widget), real-time viewing of information presentations, real-timeviewing of many user's devices' cursor pointers and locations, andcollaborative editing of information presentations.

Additional details related to the state change/time definition data isfurther described below with respect to FIG. 37.

Referring back to FIGS. 2A and 2B, the creation/playback application 120can comprise: an application load manager module 205, an editor modemodule 210, a run-time (e.g., playback) mode module 215, a collaborationmode module 220, a bus/network communications I/O manager module 225, adata loader/initial application updater module 230, a centralapplication reference and application programming interface (API)(hereinafter referred to as CARA) 235, a user input analyzer and routermodule 240, a local asset database module 245, a local stale change andtiming database 250, a time controller and manager module 255, statechange proxy module 260, multicast data broadcaster module 265, assetload manager 275, or user interface module 280; or any combinationthereof. Examples of capabilities of the system provided for by thecreation/playback application 120 are further documented in a number ofusage scenarios presented throughout this document.

It should be noted that the various components of FIGS. 2A and 2B cantogether enable the various modes of the creation/playback application120, wherein the modes of the application together with the assets canenable the capabilities of the system 100. Those of ordinary skill inthe art will see that any of the modules/databases discussed in thisapplication can be combined with any other modules/databases. Inaddition, those of ordinary skill in the art will see that certainfunctions can be separated from the modules and incorporated into otherexisting modules or new modules. Furthermore, any of the modules can beconfigured to be run on a variety of devices and/or in a variety ofdifferent modes.

The application load manager module 205 can be an initialization modulecontaining all the logic for instantiating and integrating all othermodules within the creation/playback application 120. The applicationload manager module 205 eon also be responsible for helping tocoordinate the creation/playback application's simultaneous capabilitiesusing the various modes (e.g., editor mode, run-time mode, collaborationmode, etc.). In this way, the application load manager module 205 canexecute multiple algorithms for such multi-mode capability when needed.

The editor mode module 210, run-time mode module 215, and collaborationmode module 220 can be three or more independent modules (hereinreferred to collectively as per-mode modules) that can enable thecreation/playback application instances to encapsulate, expose and/orcontain per-mode focused initialization algorithms and per-mode focusedexecution algorithms. Each per mode module can enable thecreation/playback application 120 to be run in its respective mode. Insome embodiments, a per-mode module can be run separately andindependently of any dependency on other per-mode modules, while othermodules (such as the application load manager module 205) can also usevarious combinations of the per mode modules to combine the per-modecapabilities together within the application and by users of theapplication.

The per-mode modules can receive the CARA 235 and the fullcreation/playback application context, and can populate and/or injectcustom per-mode instructions/functionality and capability into any andall dependent module instances in the application. For example, customhandling capability and routines can be provided and/or injected into:the timeline area renderer module 286 to support timeline area renderingcapability; or the user input analyzer and router module 240 to supportper mode user input handling functionality; or any combination thereof.

The bus/network communications I/O manager 225 can be responsible forall communications between the creation/playback application 120 and;external support components; external data stores; other remoteapplications (e.g., other creation/playback applications instances inuse by other users); any other network connected resource, or otherremote systems; or any combination thereof. For example, somecreation/playback application instances can be dependent on receivinglive data from ether creation/playback application instances or from thesystem 100. In addition, the bus/network communications I/O manager 225can be responsible for communications between the creation/playbackapplication 120 and; external application support systems 203; externaldatabase(s) 201; or other remote systems 202; or any combinationthereof. In some embodiments, the bus/network communications I/O manager225 can communicate using numerous protocols (e.g., TCP/IP, IPprotocols).

The data loader/initial application updater module 230 can beresponsible tor receiving information presentation data from thebus/network communications I/O manager module 22$. The dataloader/initial application updater module 230 can then add the receiveddata or references to the data to various data store modules within thecreation/playback application 120. The data loader/initial applicationupdater module 230 can also identify any state change objects that needto be executed and pass the state change objects off for execution. Thedata loader/initial application updater module 230 can also; add assetsand assets meta data to the local asset database 245; add non-immediate(future or past state change data) to the local state change and timingdatabase 250; identify state change objects which should be immediatelyexecuted upon receipt and pass these state change objects to the statechange proxy module 260 for immediate execution.

The CARA (Central Application Reference and API) 235 can provide and/orimplement structured APIs and make the creation/playback application 120addressable from any other module or component within thecreation/playback application 120. In some embodiments, any module andany state change can be invoked through access via CARA 235. In someembodiments, CARA 235 can: provide access to all application state data;organize access to some or all other modules/data/objects/instructionswithin the creation/playback application 120; or expose/implementencapsulated state change routines; or any combination thereof.

The user input analyzer and router module 240 can handle user input forall of the user interface module's submodules. The user input analyzerand router module 240 can also, independently, or in conjunction withinjected dependencies (see description of per-mode modules 210,215,220),analyze: the state of the information presentation and the informationpresentation stale or data for any given point in time, the state of thecreation/playback application 120: or the source graphical userinterface (GUI) of the input: or any combination thereof. In addition,the user input analyzer and router module 240 can respond or performactions based on such analysis. The actions it can take can include:updating/adding/deleting state changes and/or state change/timedefinition data and/or state change/time definition objects; promptingfor additional user input: identifying updated/added/deleted statechanges and/or state change objects which necessitate notification ofthe update/add/change to other components of the creation/playbackapplication 120; or notifying and providing dependent modules withchanges to state changes and/or state change objects (note thatdependent modules can include state change proxy module 260, local statechange and timing database 250, multi data broadcaster module 265); orany combination thereof. The user input analyzer and router module 240can be responsible for implementing the contextual and predictive userinterface capabilities further described with respect to FIG. 22.

The local asset database 245 can store assets and/or asset instancesalong with metadata. The local state change and timing database 250 canstore local data for any state change objects.

The time controller and manager module 253 can be responsible forimplementing/executing all time related aspects of the creation/playbackapplication 120 within information presentations. A time controller(e.g., contained within the time controller and manager module 255) canbe invoked on an as-needed basis by state change commands/objectsthrough the state change proxy module 260, such as to initialize, startor stop the time controller and/or the playback of the informationpresentation. In addition, in some embodiments the user can start orstop or activate or inactivate the time controller, and/or the playbackof the information presentation, or the time controller can be startedautomatically by the creation/playback application 120. The timecontroller can implement an internal clock management functionalitybased on the system 100's hardware or software clock capabilities anddata received from subsequent state change invocations. For example,suppose an information presentation starts with a set of predefinedstate change/time changes, contained and included in the initializationpayload processed after the application initializes. The predefinedstate change/time data can include immediate run state changes (e.g.,intended to initialize the information presentation and/or initialize orconfigure the creation/playback application 120). These immediate runstate changes can invoke the time controller with relevant datadescribing the structure of the information presentation, whereby theinformation presentation is a time oriented presentation by virtue ofthe data provided. Using this data, the time oriented informationpresentation can then be implemented and managed over time, wherein thetime controller executes the relevant state changes described in thestate change/time definition data and state change/time data objects byinvoking a state change proxy module (and passing the state change/timedata/objects) when the appropriate time in the duration of informationpresentation is reached. The time controller and manager module 255 canbe started as a routine on CARA 235 and/or invoked through and immediatestate change command/object, or started by the user or automatically bythe creation/playback application as previously described.

The state change proxy module 260 can execute all state changescontained/described in the collection of l-n state change/timedata/objects it is invoked with. Additionally, the state change proxymodule 260 can interact/query on an as-needed basis with the dependentstate change module 261. The state change proxy module 260 can also beused to implement the automatic/manual mode configurations of the system100. For example, the stale change proxy module 260 can monitor valueson CARA 235, and if it is determined that the user has entered manualmode, can prevent certain classes of state changes from beinginvoked/executed (e.g., layout change state changes).

The dependent state change module 261 can analyze a collection/list ofstate changes to be executed, and can augment the collection/list withadditional implied or dependent, state change directives, if needed.

The multicast data broadcaster module 265 can be responsible for:generating messages and/or state change/time definition data/objects forall dependent creation/playback application instances and then passingmessages/objects to the bus/network communications I/O manager module225 to be sent outside the creation/playback application 120, forexample, such as for sending real lime updates to other user'screation/playback application instances, either directly to suchinstances, or to instances external application support systems 203 forthe recipient system to determine all dependent creation/playbackapplication instances and send to all dependent instances; or anycombination thereof.

The asset load manager 275 can be responsible for all asset loadingneeds in the creation/playback application 120. As such, the asset loadmanager can monitor or listen to the local state change and timingdatabase 250 for updated/added/removed state change/time objects. Uponnotification and receipt of the updated/added/removed object, the objectcan be analyzed for references to any dependent assets referenced by thestate change object. In addition, the asset load manager 275 can providefor speed and efficiency within the system 100 and across any computernetworks used, as the asset load manager 275 can contain logic to loadassets or portions of assets for the information presentation as theassets are needed.

The user interface module 280 can comprise modules that enable an editorand/or a user to define and/or control a presentation by allowing theeditor and/or user to enter commands or provide user input through auser interface, as well as to visually represent the state of the systemand the information presentation. The user interface module 280 canaccommodate multiple approaches for defining, playing back andinteracting with an information presentation. The user interface module280 can be comprised of a stage module 281, a timeline display areamodule 282, a GUI controls module 283, a layout manager module 284 and atimeline area renderer module 285.

The stage module 281 can be the graphical area in which all elements ofthe creation/playback application 120 and information presentation arevisually displayed. The elements that can be visually displayed caninclude, but are not limited to, all graphical user interface controlsfor the creation/playback application 120 and all assets.

The timeline display area module 282 can be the graphical area in whichall elements relating to the information presentation's andcreation/playback application's state over time can be visuallydisplayed, including a timeline(s) and tracks as described below. Inaddition, the timeline display area can visually display other elementsof the creation/playback application 120 and information presentation.

The GUI controls module 283 can encapsulate and/or provide any and allnecessary graphical user interface elements and components used anywherein the user interface by other user interface modules. Part or thecontrols module 283 can be provided/fulfilled by the native support ofthe system 100 on which the creation/playback application 120 isrunning.

The layout manager module 284 can comprise many components/instructionsets for providing various layouts for the assets within thecreation/playback application 121 or on the information presentationscream(s). The layout manager module 284 can provide flexible, inputdriven algorithms for laying out l-n assets in various positions. It canalso encapsulate and provide for automatic or convenience layoutcapabilities with additional encapsulated algorithms with predefineddefault values or which can evaluate the context of the assets which itis laying out and automatically determine the best layout. With respectto evaluation or context and subsequent automatic layout, the layoutmanager module 284 can analyze various aspects of the system 100,including, but not limited to: number of assets to be laid out, type ofassets, dimensions and/or aspect ratio of assets to be laid out (e.g.,8.5 inch by 11 inch document, 800 pixel by 600 pixel presentationdocument, 4:3 video document, 16:9 video document, etc), font size(s) oftext within the assets, or available space on the user interface withincreation/playback application 120 or information presentation on theparticular device that is being used for display; or any combinationthereof. For example, depending on the configuration of the application,the layout manager module 284 can determine that two assets are to bedisplayed (e.g., a 4:3 video, and an 800 by 600 presentation), and thendetermine that there is enough available space to layout the assets sideby side on the screen. Likewise, the layout manager module 284 coulddetermine that there is not enough space to optimally display bothassets, and provide a side by side layout which only can show one assetat a time but can allow the user to “slide” back and forth between thetwo assets. In this scenario, it could also determine to show the twoassets as one on top of the other.

The timeline area renderer module 286 can be responsible for renderingrelevant state change and time definition data visually in the timelinearea(s) and attaching GUI behavior/interaction handlingcode/instructions to the timeline area(s). The timeline area renderermodule 286 can: contain core and generic subroutines used in variousmodes as well as routines intended for a per mode basis and/or beinjected with additional per-mode subroutine codes in order to renderthe timeline area.

Various Modes

FIGS. 15-16 outline some of the various modes enabled within andsupported by the creation/playback application 120. As indicated in FIG.15, the editor mode capabilities, enabled in one embodiment by theeditor mode module 205 can enable a user (e.g., editor) to define aninformation presentation over time. (It should be noted that a user(e.g., editor) can include any person or computer capability that usesany device for defining the information presentation.) The defining ofthe information presentation can include, but is not limited to,defining: the assets that are presented and the times at which theassets arc presented; the way the information presentation and/orasset(s) is presented over time; the interactive and functionalcapabilities of the information presentation and/or assets as displayedover lime; the interactive and functional capabilities over time of thecreation/playback application 120 for other users using or viewing theinformation presentation; any other aspect of the stale of theinformation presentation or assets for any given point in time; or anyother aspect of the state over time of the creation/playback application120 for other users using or viewing the information presentation; orany combination thereof. This capability will be referred to hereinafteras “defining the state of the information presentation over time.” Inone embodiment, the information presentation state can be defined for atleast one selected point in time (e.g., on at least one timeline).

In one embodiment, the user can interact with the at least one timeline,the timeline being rendered visually by the user interface module(s), todefine the information presentation, wherein the creation/playbackapplication can create state change/time definition data fen the definedstate changes that comprise the defined information presentation. Anadditional editor mode capability can include the ability for thecreation/playback application 120 to enter a record mode. While therecord mode is activated, a user can make state changes to aninformation presentation and assets in real-time and thecreation/playback application 120 can monitor and record all input madeby the user as it relates to state changes of the informationpresentation, assets, creation/playback application 120, etc. Somespecific examples of recordable state changes include, but are notlimited to: actuating, activating, moving, showing, hiding assets:taring the whiteboard widget; editing a document using the documentediting widget; editing a spreadsheet using the spreadsheet editingwidget; drawing using the drawing suite widget; motoring with a device'scursor pointer; or manipulating or interacting with any assets orwidgets; or any combination thereof. After a record mode session isended, the recorded state changes can be rendered in the timeline area.The user can repeatedly engage the record mode to either overwriterecorded state change/time definition data or to supplement it withadditional recorded state change/time definition data. Recorded statechange/time definition data can be immediately edited using the othereditor mode capabilities, such as interacting with the timeline area.Additionally, the creation/playback application 120 can persist theinformation presentation data and it's constituent parts (i.e., to anexternal database 201, a remote system 202, an external applicationsupport system or sub system 203), and recall them again for editing aswell as playback at any time. It should be noted that an editor user caninclude any person or computer capability that uses any device fordefining the information. It should be noted that the record modecapability of the system is recording actual changes to the underlyingdata and state of the creation playback application and the informationpresentation such that that information presentation and full sequenceof events may be “played back” at a later, rather than recording thescreen or a video.

As indicated in FIG. 16, the runtime/playback mode capabilities, enabledin one embodiment by the runtime/playback mode module 215, can enable auser to: access and view information presentations and assets; interactdirectly and indirectly with the information presentation and assets;customize and edit the information presentation and the assets and theway they are presented; interact with other users of the system; orcollaborate with other users of the system together with the informationpresentation or the assets; or any combination, thereof. This capabilitywill be referred to hereinafter as “the user viewing and interactingwith the information presentation”.

The collaboration mode capabilities, enabled in one embodiment of acollaboration mode module 220, can enable users to collaborate inreal-time and non real-time in ways that may not be directly supportedthrough any disclosed assets. For example, users of the system could usea collaboration mode capability whereby all users are able to view anyother user's interactions or user invoked changes to the state of theinformation presentation, assets or application. Another example couldbe where each user can view the cursor indicator or mouse pointer or allother users viewing an information presentation collaboratively(hereinafter called a multi-cursor view). In this example, amulti-cursor view is not necessarily provided for by any other module,and thusly could be provided for by the collaboration mode module 220.When the multi-cursor view is activated the collaboration mode module220 can send state change/time definition data through the multicastdata broadcaster module 265 to notify the other users' applications toalso activate this feature immediately. After all users' applicationshave the feature activated, each of the respective collaboration modemodules 220 can then be responsible for (e.g., while the feature isactivated) engaging the multicast data broadcaster module 220 to sendthe coordinate details of the respective users' cursors as statechange/time definition data.

Those of ordinary skill in the art will see that any of thecapabilities/functions of any of the modes can be combined with thecapabilities of other modes.

FIGS. 40A and 40B describe some of the graphical user elements andoperations that can be available to a user within the various modes ofthe system, such as for interacting with the information presentation ordefining the information presentation state over time. (It should benoted that in some embodiments the user interface features can also beused outside system 100.) 1) displays an embodiment through which a usercan change/edit the information presentation, specifically, add/removeassets be displayed, change the information presentation or stagelayout, and switch between automatic mode and manual mode. 2)specifically details an embodiment for selecting a layout. 3)specifically details a method in which a user can locate availableassets. 4) specifically details a user interface control for switchingor toggling to automatic mode from manual mode (detailed below). 5)details other ways in which the user can manipulate the layout or theassets within the information presentation; interact with theinformation presentation or assets or define the informationpresentation state or state of the assets over time.

By manipulating the information presentation during playback, such asselecting/configuring/changing the layout or adding/removing assets fromthe screen, or inviting other users to collaborate, thecreation/playback application 120 can change from playing theinformation presentation in its predefined format (automatic play mode)into manual play mode, wherein the creation/playback application 120 canno longer implement some or all of the predefined informationpresentation's state changes, such as the display/activation of assetsand layout changes. The user/editor can immediately switch back toautomatic mode with a single user gesture, such as clicking on a button.This capability is also detailed in FIGS. 5 and 27-28, and theiraccompanying descriptions below.

Tracks

The user interface module 280, together with other modules, canaccommodate multiple approaches for defining, playing back, visualizingand interacting with an information presentation. In one embodiment, oneof the multiple approaches that can be implemented using the userinterface module 280 can be the implementation by the creation/playbackapplication 120 of a visual timeline with zero or more tracks within atimeline display area, wherein the visual timeline and tracks are aspecific graphical user interface that can be used interactively by theuser with the system to: a) define states of the informationpresentation, states of the assets, and states or the creation/playbackapplication 120 and/or the changes of the state of the informationpresentation, the state of the assets, and the state of thecreation/playback application over time: b) interact with theinformation presentation, the assets, and the creation/playbackapplication's states and state changes over time; or c) generallyinteract with the information presentation, the assets, and thecreation/playback application; or any combination thereof). Thegraphical user interface can also be used to visually represent for theuser the information presentation, the assets and the creation playbackapplication (e.g., changes of the state over time).

Tracks can flexibly accommodate the user in various capacities invisualizing, interacting with and configuring/defining an informationpresentation, its assets and the creation/playback application's stateover time. The timeline area can have multiple tracks all certain tracksdevoted to visualizing or configuring certain aspects of the state overtime. For example, a multi-track embodiment in any mode may contain atrack for each of the l-n assets to be displayed or activated/actuatedand certain aspects of state related to that specific type of asset.Tracks of this nature can be referred to as asset tracks. In additionthere can be tracks that can be specifically implemented for definingand controlling other aspects of the information presentation over time,such as the layout of the information presentation, the abilities orconstraints of the user to modify the layout of the informationpresentation, the functional capabilities of the assets and theinformation presentation, or the logic and rules regarding if real-timeinteractive widgets should be displayed and provided; or any combinationthereof. Tracks of this nature can be referred to as control tracks.Examples of control tracks include: asset layout control tracks purposedaround controlling/defining the layout of the assets; application layoutcontrol tracks purposed around controlling/defining the layout of thecreation/playback application; user constraint tracks purposed arounddefining certain actions that can or cannot be taken by the user; or anycombination thereof. Additional detailed information with regards totracks can be found in FIG. 21 and its accompanying description.

Assets and Structure

FIG. 3 sets forth the assets, which, as indicated above, can comprisenon-interactive assets 305 and/or interactive assets 310. Thenon-interactive assets 305 (which can be assets that are not interactivewith either another person, system 100, or another system) can comprisereal time assets 315 and/or non real-time assets 320. The assets can behuman and/or machine readable. The real-time assets 315 can comprise:real time information streams, real-time audio streams, or real-timevideo streams, or any combination thereof. The non real-time assets 320can comprise: at least one time-based file (e.g., audio file, videofile, multimedia file), or at least one document file, at least oneimage file (e.g., picture), or any combination thereof. The interactiveassets 310 (which are assets that can be interactive with anotherperson, system 100, or another system) can comprise widgets (e.g.,components) that can: gather user input, provide dialogue with the user,provide interactivity between the user and the information presentationand/or assets, provide interactivity between the user and the system 100and/or another system, or provide interactivity and collaborationbetween multiple users or the system 100 and/or another system, or anycombination thereof. All of these capabilities will be referred tohereinafter as “widget interactivity.” It should be noted that, in oneembodiment, non-interactive assets can become interactive assets bybeing loaded into widgets. Various examples include: a text documentbecoming editable by being loaded into a document editing widget, anaudio or video document becoming editable by being loaded into anaudio/video editing widget, or a live video stream becoming configurablewithin a video playback or audio/video chat widget, or any combinationthereof.

The widgets can comprise at least one text chat widget, at least oneaudio/video chat widget at least one asset browser widget, least onediscussion widget, at least one audio recording/playback widget, atleast one video recording/playback widget, at least one stream playbackwidget, at least one audio/video editing widget, at least one documentviewing widget, at least one document editing widget, at least onelinking widget, at least one activity widget, at least one slide showediting/viewing widget, at least one diagram editing/viewing widget, atleast one drawing suite widget, at least one whiteboard widget, at leastone polling widget, at least one survey widget, at least one assessmentwidget, at least one spreadsheet widget, at least one user list widget,at least one source code viewing and editing widget, at least onetracking widget, at least one glossary widget, at least one screensharing and/or screen recording widget, at least one wiki widget, atleast one bookmark widget, at least one calculator widget, at least onepresentation index widget, at least one layers widget, at least oneintegrations widget, or at least one third party platform interactivewidget; or any combination thereof. It should be noted that any widgetcan be provided using third-party software. In addition, those ofordinary skill will see that any widget function(s) can be combined withany other widget function(s) to form other widgets.

The text chat widget can enable real-time text-based into discussionbetween multiple users of the system or the information presentation.

The audio/video chat widget can enable real-time audio and video chatbetween users of the system through their respective devices, wherebyeach user can simultaneously view real-time video and audiofeeds/streams being broadcast from many users of the system. In someembodiments, it may use the stream playback widget. Various video cameraand audio input sources can be supported through standard technologiesand interfaces on various devices. Examples include web cam technologybuilt into many notebook computers and video camera support built intomobile device. In addition, the audio/video chat widget can support arecord mode and can used to record a uses audio and/or video input foruse in a variety of ways in non real-time scenarios. For example, theaudio and/or video recorded could be used as an asset within aninformation presentation or within any other widget, such as being usedin an audio and/or video mode embodiment of the discussion widget.

The document editing widget can enable a user to create, edit, saveand/or recall and view documents and/or can provide rich text editingsimilar to a document editor (e.g., Microsoft Word, etc.).

The slide show editing/viewing widget can enable a user to design,create, edit, save, recall or view (or any combination thereof) slideoriented documents (e.g., slide show presentations) and slide showssimilar to other slide show presentation emit on tools (e.g., MicrosoftPowerPoint, Apple Keynote, etc).

The linking widget can be used independently and standalone or inconjunction or as part of any other asset or widget. The linking widgetallows any other assets or object within an asset or widget, or withinrich text element (e.g., image, video, drawing/diagrams element such asa shape, highlight, bitmapped image, etc; discussions within thediscussion widget) or any text within any asset to be linked to orcontain links including, but not limited to: 1) one or more points intime in the information presentation; 2) other assets; 3) other statesof other assets; 4) other layers in the layers widget; 5) otherinformation presentations; or 6) any other URI or URL; or anycombination thereof, any of which can be displayed within the linkedfrom/original information presentation or outside of the informationpresentation.

The activity widget can be used in conjunction with other widgets inorder to assemble or connect a sequence of other widgets or a sequenceof layers or screens containing widgets together into a multistepinteractive user activity, wherein such activity can be added to andbecome a part of an information presentation. Further, the activitiescan be timed or time limited and can be required or not required to becompleted by the user.

The diagram editing/viewing widget can enable a user to design, create,edit, save and/or recall and view various types of diagram orienteddocuments similar to other diagram creation and viewing tools (e.g.,Microsoft Visio, etc).

The drawing suite widget can enable a user to design, create, edit,save, recall, insert or view (or any combination thereof) various typesof image files and assets of various types including raster/bitmapformats (e.g., GIF, PNG, JPEG) vector file formats (e.g., SVG, EPS,Adobe Illustrator), various image or vector objects or drawing shapes(i.e., circles, squares, highlights, text,). The drawing suite widgetcan support both vector drawing and image manipulation capabilities andraster/bitmap image manipulation and capabilities, and can combinecapabilities of various other standard drawing and image manipulationtools (e.g., Adobe Illustrator, Adobe Photoshop, Adobe fireworks). Asmay be the case with any widget, the drawing widget may beplaced/moved/positioned over top of any other asset or over all assetsdisplayed within the creation/playback application. It is important tonote that any widget can be placed on top of any other widget within theuser interface and stage of the creation/playback application or withina layer (i.e., within the layer widget) or any combination thereof.

The asset browser widget can enable a user to browse, preview, view, orinteract with (or any combination thereof) any asset that may or may notbe related to a given in formation presentation. The asset browserwidget may use other widgets if necessary in order to provide itscapabilities related to browsing the assets. In addition, the assetbrowser widget can be one way in which a user may select and add assetsto an information presentation or to the stage.

The discussion widget can provide various ways for a user to engage indiscussions with other users of the system, according to severalembodiments. The discussion widget can be used tor various functions formessage oriented communication, such as posting, replying, viewing, orarchiving messages (or any combination thereof). The discussion widgetcan be configured to support discussions in several ways, including:ongoing and/or thread oriented discussions similar in structure to otherwidely used online discussion/message boards (also known as forums);ongoing lists of comments similar in structure to other widely usedcomment systems; or in other ways (including question and answer, etc.).The discussions and posts/messages can be text or rich text based (i.e.,text with various font sizes, colors, embedded drawings, embedded videoand/or multimedia, embedded images, etc. examples of which could beMicrosoft Word documents or HTML documents), as well as audio and/orvideo based. For example, discussions, messages, discussion posts andcomments can be posted by users in an audio and/or video format, oraudio and/or video in rich text format, wherein such audio and/or videomay have been recorded using the audio/video chat widget's record modecapability Discussions and messages can be associated with a given pointor several points in time within an it presentation, and as such, can berendered visually on the timeline in various ways. The discussion widgetcan be configured to notify a number of devices when an interaction withthe discussion widget has happened (e.g., notifications to mobilephones, SMS messages, email, notification popups in the creationplayback application, when messages are posted or replied to). Inaddition, the discussion widget and its constituent posts/messages andthe posts/messages content can provide for access (such as throughlinks, etc.) directly into the information presentation, at the exactpoint in time for which the discussion or posts/messages can beassociated or other points in time in the information presentation. Inaddition, the discussion widget and its constituent post/messages canboth be linked as well as contain links in any way provided for by thelinking widget, and both can also easily reference and be linked toother discussions and posts/messages.

FIGS. 48A and 48B illustrate an example of a specific configuration ofthe discussion widget, according to one embodiment. In this example, thediscussion widget is configured in a question and answer style. In 1, auser (referred to as Julie) is watching an information presentation,created by another user (referred to as Professor X). Julie has opted toactivate the question/answer display on the information timeline. In theexample, various questions and answers related to the informationpresentation are represented on the timeline by yellow pins, placed atspecific points in time on the timeline that the questions/answers arerelated to. In 2, while Julie watches the presentation, she finds thereis a topic that she does not understand and has a question regarding it.She points her mouse over the 1 or more pins that may appear on thetimeline near the portion of the information presentation she iswatching. On pointing her mouse over each pin, questions/answers relatedto the specific point in time the pin is representing can appear in awindow using the preview portion of the question and answer configureddiscussion widget. In 3, Julie cannot find any questions and answersrelated to her own question, and she decides to activate the searchwidget to search other course content such as discussion boardsregarding her topic. In 4, Julie's search produces no results, so Julieactivates the “ask a question” portion of the question and answerconfigured discussion widget, upon which a dialog box appears, whereJulie enters and submits her question. In 5, Julie's question gets sentto multiple locations, including the discussion board, and as an emailto Professor X's email and mobile phone. Embedded in the email is a linkto the exact moment and time of the information presentation that Juliehas a question about, so users can click on it and review the content inthe context of Julie's question. In 6, Professor X arrives at his officethe next day, he notices that he has received an email question fromJulie. He opens up the email, reads Julie's question and clicks on theembedded link. Professor X reviews the portion of the presentation thatJulie has a question about and responds through the question and answerconfigured discussion widget by pressing on the “Answer” button. In 7,Professor X feels that Julie's question is a very good one, and wants toarchive the question and his answer into the information presentation,so that it can be previewed using the preview portion of the questionand answer configured discussion widget related to the timeline of theinformation presentation. In 8, the archived question and answer nowappears represented in the information presentation on the timeline atthe point in time that the original question was asked. This “archives”the question and answer for future viewers of the informationpresentation to reference at that point in time of the informationpresentation.

The audio recording/playback widget can allow for recording of at leastone audio document and/or the selection of at least one audio document,or controlling, viewing or hearing the playback of the audio document,or any combination thereof. The video recording/playback widget canallow for recording of at least one video document and/or enable a userto select at least one video document, or control, view or hear theplayback of the video document, or any combination thereof.

The stream playback widget can allow a user to select and/or to connectto at least one real-time (e.g., live) audio, real-time video, orreal-time data stream, or to view and/or hear the streams, or anycombination thereof.

The audio/video editing widget can enable a user to flexibly edit audioor video documents and assets or any combination thereof in a non-linearfashion similar to other non-linear editing systems. Similar to othernon-linear editing systems, a user can create, define and output/savevideo and audio files and/or assets that are composited or mixed frommultiple video and audio files/assets in a non-destructive manner (e.g.,without destructing, degrading or modifying the original video and audioassets). The user can use many standard user interface elements andapproaches similar to the user interface of other non-linear video andediting systems, such as layers, visual in/out points, etc.

The document viewing widget can enable a user to load and view textbased documents in various formats (e.g., Microsoft Office, PDF).

The whiteboard widget can enable a user to draw and diagramindependently, above, and/or in conjunction with (e.g., on) any otherasset, individually (e.g., visible to only one user) or collaboratively(e.g., with other users in real-time (e.g., live) and non real-time(e.g., non-live)). In addition to its own native drawing capabilities,the whiteboard widget can make use of other widgets, such as, but notlimited to, the drawing widget and diagram editing and viewing widget.

The polling widget can enable a user to define and/or participate inquestion based polls, both in real-time and non-real time. The surveywidget can enable a user to define and participate in surveys, both inreal-time and non-real time.

The assessment widget can allow users to define orview/access/participate in assessments (such as quizzes and exams) usingnumerous question types (e.g., multiple choice, true/false, branching,matching, drag and drop, short answer, essay, etc.) and at least onequestion. The assessment widget can be configured to present to the userone or more questions or assessment activities at any paint during aninformation presentation. The assessment widget can be configured invarious ways including presenting any or all questions/assessmentactivities modally (e.g., where the user must provide an answer or acorrect answer before proceeding within the information presentation).The assessment widget can also allow determining, storing and/ordisplaying scores, both within an information presentation and acrossmultiple information presentations. Assessments created/defined usingthe widget can be timed or time limited and individual questions in theassessment can be timed or time limited. The assessment widget can alsobe configured in various ways to ensure to ensure security of theassessment data, authenticity of the user and the integrity of theassessment. For example the assessment widget can secure a users devicesuch that only the creation playback application (and no otherapplications or limited access to other applications) can be used duringan assessment or track usage of any other applications on the device(e.g., both capabilities can be used to prevent cheating, etc.).Additionally, the assessment widget can use or be used in combinationwith various real-time assets and widgets to ensure integrity or theassessment and authenticity of the user and to prevent cheating. Forexample, the audio/video chat widget can be used to verify theauthenticity of the user and to proctor and/or monitor users as they usethe application and participate in an assessment.

The spreadsheet widget can create/define/edit and/or allow viewing ofspreadsheets (e.g., Excel) and/or spreadsheet oriented activitiessimilar to other spreadsheet applications (e.g., Microsoft Excel, etc.).

The user list widget can allow for users of the system or informationpresentation to see information about other users of the system orinformation presentation, and/or interact with other users or solicitinteractions with other users of the system or of the informationpresentation. For example, a user could activate the user list widgetand see a list of all users (e.g., editors, viewers, contributors) of aninformation presentation as well as additional information, such astheir name, online status (or last time online), availability. A usercould also interact with other users, such as “knocking” on their dooror requesting various forms of collaboration, such as chat,collaboration mode, shared white board, etc.

The source code viewing and editing widget can enable a user withsoftware development tool capabilities, ranging from enabling a user toformatted and color coded computer programming source code (e.g., sourcecode formatting) to enabling a user to participate in a fully integrateddevelopment environment. In the integrated development environmentembodiment, the source code viewing and editing widget can provide forcapabilities such as, but not limited to: source formatting, codehighlighting, editing of source code, code completion and hinting, steporiented debugging capabilities, linking to object code, or compilingand execution of code, or any combination thereof. In addition, thesource code viewing and editing widget can be one way of implementingand providing for programming to CARA 235. This ability to programagainst and for CARA 235 could be used to define an informationpresentation and the state of the information presentation and assetsover time programmatically. For example, a user could choose to writecode for or “script” to define an information presentation and the stateof an information presentation over time or a portion of an informationpresentation or any asset at any point m time in an informationpresentation, as an alternative to or in conjunction with using theapplications various other capabilities (e.g., user interface) availablefor defining an information presentation.

The tracking widget can enable a user to track and/or view his progresswithin an information presentation and across information presentations.Further, it can track and log every single action or interaction a usertakes and the time at which it happened, providing for an analysis andreporting data set. The glossary widget can enable a user to view and/oraccess a glossary for a specific information presentation, or acrossinformation presentations. The screen sharing and screen recordingwidget can enable a user to replicate and/or broadcast in real-timeand/or record or playback (e.g., in real-time or non-real time) at leastone portion of a user's device's screen, and/or to view another user'srecording and/or broadcast on other user's screen. The screen sharingand screen recording widget also allows users of the system to takecontrol of other user's systems (i.e., operating systems) such that auser or multiple users can control another user's computer. In addition,the recorded video of the screen capture can be used as an asset in aninformation presentation or within other assets.

The layers widget can provide a user and/or editor with the ability tocreate and edit multiple layers upon the stage within thecreation/playback application. Each layer operates similarly to thestage and can be used to display one or more assets. From the userinterface of the creation/playback application 120 users can choose toview the one or more layers containing the assets together orseparately, depending on their capabilities, as allowed/disallowed by aneditor. (Details regarding constraint controls discussed later in thisdisclosure).

The wiki widget can provide a user with wiki oriented collaborativeknowledge building functions similar to other wikis using rich text,such as Wikipedia. The bookmark widget can enable a user to bookmarkand/or recall any point in time in an information presentation or anasset at any point in time, and/or to share and or send these bookmarkswith/to other users and other systems. The calculator widget can enablea user to display a graphical calculator (including various types ofcalculators, such as standard, accounting, scientific, binary, graphing,etc.) and or to define calculator-based exercises or activities to heused in an information presentation. Those of ordinary skill in the artwill see that various calculators (e.g., scientific calculator,accounting calculator, etc.) can be used. The presentation index widgetcan enable a user to define and/or access an index of various points intime in art information presentation.

The third party platform widget can enable a user to integrate/usewidgets built or compiled for/from a variety of platforms. The thirdparty platform widget can include, but is not limited to, a widget builtfor the Adobe Flash platform ((SWF files etc). Microsoft SilverLight, orother technologies or web technologies (e.g., HTML, JavaScript, CSS,etc.), used separately or in combination.

FIG. 37 illustrates the nature and structure of the informationpresentation data that comprises the general input/output for thecreation/playback application 120, according to one embodiment. As notedin FIG. 37, information presentation data can be comprised of theplurality of assets, the assets metadata, other metadata, and statechange/time definition objects and data. The state change/timedefinition objects and data can contain information about change in thestate change of the information presentation and the time at which thestate change should be executed, including, but not limited to: the waythe information presentation and/or assets are presented; the date/timethat the state change occurred; the user or user id or the user whoinvoked/created implemented the state change; any of the data values ordata associated with an asset; or the interactive and functionalcapabilities of the information presentation or assets, over time or ata given time, or any combination thereof. The state change/timedefinition objects and data can be of two natures: pre-defined orreal-time. The predefined state change/time definitions can be createdby an editor using an editor-mode embodiment of the system and thensaved to be recalled and used by either an editor or user at a latertime using the system. Together, a collection of pre-defined statechange/time definition objects can form and define a predefinedinformation presentation over time. Real-time state change/timedefinition objects can be potentially received throughout the use of theapplication. State change/time definition objects of both natures (e.g.,predefined, real-time) can define state changes of the informationpresentation or the system and can be: (1) executed at a previous pointin time in the information presentation (e.g., should the user(s) of thesystem be able to/allowed to and choose to backtrack in time over thedefined information presentation); (2) immediately executed; or (3)executed at some point in the future or ahead of the current time withinthe information presentation; or any combination thereof.

Each state change/time definition object can generally contain a timevalue, which can be the time in milliseconds for the state change to beexecuted. The time can be either absolute time or time relative toanother time or relative to the start time of the presentation. Inaddition, the state change/time definition object can comprise anindicator set to a value indicating a state change is to be executedimmediately upon receipt. Furthermore, the state change/time definitionobject can comprise a relative-to indicator, which can indicate what thetime value is relative to.

The state change data within each state change/time definition objectcan define an actual change in state to the information presentation, achange in state to the creation playback application 120, or a change instate to other state change/time definition objects. In one embodiment,each state change/time definition data can be: a discrete unit with anamed reference or memory address based reference to a data values inmemory within the application context and a new value to change to; adiscrete unit with a named or memory address based reference toexecutable instructions along with input values for the executableinstructions; or a discrete unit with executable instructions along withinput values for the executable instructions; or any combinationthereof.

Methods of Using the Creation/Playback Application

FIGS. 38A and 38B illustrate a method by which one embodiment of thesystem or application can initialize, listen for and process incomingstate time data. In 3805, a user of the system can request to view,access, and/or edit an information presentation. In 3810, a servercomputer 110 can allow access to the creation/playback application 120.This can be done as noted in 3811 and 3812. In 3811, the server computer110 can locate and provide to the user the information presentation datato be loaded. In 3812, the server computer 110 can create an instance ofthe creation/playback application 120. In 3815, the user's clientcomputer 115 (e.g., host system such as device, etc) can load thecreation/playback application instance and initial informationpresentation data payload. In 3820, the user's instance initializesitself and its graphical user interface within the host system. This canbe done as noted in 3821, 3822 and 3823. In 3821, the support frameworkis initialized. In 3822, the user interface framework is initialized. In3823. the creation/playback application 120 components are initialized.

In 3825, the creation/playback application 120 can open a networkcommunication connection to the server computer 110 using a bus/networkcommunications I/O manager 225. Remote server access can be connectedvia any IP/Socket Protocol. In 3830, the creation/playback applicationinstance can notify the server computer 110 that the initialization iscomplete. In 3835, the creation/playback application instance canprocess incoming data related to the initially received informationpresentation data. This can be done by invoking the data loader/initialapplication updater module 230 with the initialization payload data, asset forth in 3850-3855.

In 3850, the information presentation data can be received. In 3851, theassets and assets metadata can be added/updated to the local assetsdatabase 245. In 3852, the real-time or other state change/timedefinition objects that require immediate processing and execution canbe separated and identified. In 3853, these real-time or other statechange/time definition objects that require immediate processing andexecution can be handed to the state change proxy module 260 to beimmediately executed. In 3855, the state change/time definition objectscan be added and/or updated in the local state change and timingdatabase 250 with other state-change/time definition objects which willpotentially need to be executed later.

In 3840, the creation/playback application instance can begin theprocess of continually listening for incoming information presentationdata, as set forth in 3842-3844. In 3842, the creation/playbackapplication 120 listens for newly received data. In 3842, it isdetermined whether new data is received. If yes, new data is received,in 3843, the data is sent to be processed (as described and illustratedin 3835). If no, new data is not received, in 3844, the process returnsto 3842, where the creation/playback application 120 continually listensfor new incoming data.

FIG. 4 illustrates a method 400 for defining information in aninformation presentation, according to an editor mode embodiment. In405, an editor can accept assets to use for the informationpresentation. In 410, the editor can define the state of the informationpresentation. In 415, the editor can save the state of the informationpresentation. In 420, an editor can decide to edit the state of theinformation presentation. If the editor decides to edit the informationpresentation, the process can return to 405 or 410 and repeat. If not,the process can end.

FIG. 5 illustrates a method 500 for presenting information, according toa run-time mode embodiment. In 505, an information presentation can playin automatic mode. In 510, it can be determined whether a user wants tochange to manual mode. This can be done, for example, by recognizingthat the user has clicked on a certain button, clicked on the corner ofan asset, or by another method. In one embodiment, the ability to changethe information presentation (e.g., go from automatic to manual mode)can be allowed or disallowed by an editor. If yes, in 515, theinformation presentation plays in manual mode. If no, the informationpresentation returns to 505 and is played in automatic mode. In 520, itis determined whether the user wants to change to automatic mode. If no,the information presentation is run in manual mode, as indicated in 515.If yes. the information presentation is played in automatic mode, asindicated in 505.

Non-Track and Single Track Embodiments

FIGS. 6-14 and 17-18 illustrate a single timeline track embodiment. FIG.17 illustrates a single track timeline containing a single display trackand representing, using various indicators, various aspects about theinformation presentation over time, according to one embodiment. In1705, a basic timeline indicates the duration of an informationpresentation. In 1710, a timeline with two data layers (which can becolor coded) is illustrated. The white and gray area's can represent oneaspect of the information presentation's state changes over time. In1715, third and/or fourth additional aspects of the informationpresentation's state changes over time are illustrated by pins. In 1720,two additional aspects of the information presentation's state changesare represented: one aspect is represented by the asterisk symbol, andanother aspect is represented by the dark line on the bottom of thetimeline. In 1725, a sample single track timeline is shown with examplesof aspects of an information presentation's state changes as they couldbe displayed. In 1, a timeline indicating the duration of theinformation presentation is shown. In 2, a pin (which can becolor-coded) can indicate the addition of an asset. In 3, alternatingcolors on the timeline can indicate asset transitions (e.g., such aspage changes in a slide show document or pages in a text baseddocument). In 4, the asterisk symbol can indicate an asset being removedfrom the stage. In 5, a progress tracking bar can indicate progress andlocation through the information presentation. In 6, a different shapedpin (which can also have a different color) can indicate a quiz orexercise for the user to complete in the information presentation.

FIG. 18 illustrates several basic single track layout examples. In 1805,a basic single track timeline with one asset is shown. In 1810, a singletrack timeline is shown. In the first screen, one asset is shown. In thesecond and third screens, two assets are shown at the same time. In1815, another single track timeline is shown. In the first screen, oneasset is shown. In the second and third screens, two assets are shown.In the fourth screen, three assets are shown. In the fifth screen, fourassets are shown.

FIGS. 6-8 illustrate screen shots where an information presentation canbe defined according to an editor mode embodiment. In theseillustrations, a user has entered the creation/playback application 120in editor mode and has (already) indicated to the system that theinformation presentation will be comprised using 1 video asset and 1slide presentation document asset and has chosen the 1 of the eachspecific assets, slide presentation document and video.

FIG. 6 illustrates the following: a play/pause button 605 which cancause an information presentation to start playing or be paused; a timeindicator 610, which can indicate the time in a informationpresentation; timeline area 615, which can indicate and allow fordefining the various elements of an information presentation or thechange of an information presentation over time; create button 625,which can be used by a user to create a new state change in theinformation presentation; save button 620, which can be used to save theedited information presentation; and a number area 630, which can allowan editor user to enter a number of new state changes to create inconjunction with the create button 625.

FIG. 7 illustrates a screen shot where an editor has chosen to definethe state of the information presentation at time 0:00. Thisillustration shows a newly created information presentation with asingle state change defined and displayed by a state change indicator730, which appears here on the timeline. In this illustration, the userhas clicked/selected the state change indicator, and the state changeeditor panel 710 has appeared, allowing the editor user to definevarious slates pertaining to the type of information presentation beingcreated, as defined by the use of the particular assets (in thisscenario, 1 video asset and 1 slide presentation). Using the layoutstate selector 715, the editor user can select from a predefinedselection of layout states for the video and slide document at the givenparticular point in time that the editor panel is being displayed for.in this case, 0:00. In this embodiment, the editor user can adjust thetime of the state change in several ways. By clicking on or in anotherway selecting the state change indicator on the timeline with a pointingdevice and then dragging the state lime change indicator to the right orleft in the timeline, the editor user can adjust the time within theinformation presentation that the state change is executed. As analternative to the select/click drag method, the editor user can chooseto manually enter the time 720 that the state change is executed. In735, the user can set the state of the information presentation for thegiven time in the way of which slide of the slide document is presentedand displayed by clicking the up or down arrows or manually entering thenumber of the slide to be displayed into the text box. While interactingwith 735, the slide to be displayed can be previewed using the slidepreviewer 740. There is also a delete button 725 that can be used todelete the state change from the information presentation and thetimeline area.

FIG. 8 illustrates another screen shot example illustrating many statechange indicators which an editor user has created and which caninteract with and signify a number of state changes at various timesthroughout the information presentation as indicated by 810.

FIG. 9 illustrates a screen shot according to a single track runtimemode embodiment, showing how a user can view/playback an informationpresentation. In this example, the information presentation uses onevideo asset and one slide document asset throughout the entire duration.The timeline 905 can have alternating colors to indicate certain aspectspertaining to the change of state of the information presentation overtime. In this example, the alternating colors represent when a new slidefrom the slide document asset will be displayed. Options 910 can allowthe user to control or customize the state of the informationpresentation at any time. In 910, die user can select from differentpredefined layouts for the two assets that arc to be displayedthroughout the entire duration of the information presentation.

FIGS. 10-12 illustrate the information presentation in FIG. 9 in thevarious layout states, which may be customized by the user using options910. FIGS. 10-12 are screen shots illustrating the informationpresentation at a later point in lime. The time within the informationpresentation is 19:16, as is indicated by the time indicator on thetimeline, and the slide from the slide document asset that is programmedto show during this time is illustrated.

FIG. 13 illustrates a screen shot, according to non-track runtime modeembodiment. The assets shown at the specific time in the displayedinformation presentation are: 1305 a video file, 1315 a document editingwidget, 1310 an assessment widget, and 1320 a calculator widget. 1325illustrates a next and previous button which can enable a user tonavigate forward and backward through an information presentation, suchas, but not limited to, enabling a user to jump forward and backward toprevious and next points in time in an information presentation whenupcoming and prior assessment questions arc displayed. 1330 illustratesa selection of options which can allow the user access to various meansof configuring the information presentation. Those illustrated includebuttons for showing and hiding various assets and widgets, including theshown document editing and calculator widgets and the not-visible chatand discussion widgets. 1335 shows various other options for controllingor configuring the information presentation, including: a play/pausebutton for playing and pausing the information presentation, a timeindicator for indicating the time in the information presentation, aclosed captions button for showing/hiding closed captions which canaccompany the information presentation, and a full screen button whichcan allow the information presentation to be viewed in full screen.

FIG. 14 is a screen shot illustrating a single track runtime modeembodiment. This screen shot shows a selection of assets that can besynchronized and/or viewed together within an information presentation.The assets are: a video file, a document viewing widget, and an assetbrowser widget. The video of the professor is synchronized with thedisplay of pages in the displayed document, as well as with the displayand actuation of other assets and widgets over time in the informationpresentation. In addition, the user can access other assets if needed,using the asset browser widget.

Multi-Track Embodiment

FIGS. 19-34 illustrate features of a multi-track embodiment. FIG. 19illustrates an icon legend. FIG. 20 compares the single track embodimentwith the multi-track embodiment, showing that both embodiments similarlyrepresents the same set of one or more states of the informationpresentation and/or the changes of the state of the informationpresentation over time. The first timeline illustrates a single tracktimeline collapsed view, which shows different states of a presentationon a single track timeline. The second timeline illustrates amulti-track timeline, where different states of an informationpresentation are shown on a timeline that has details related to severaltracks. In both embodiments, an editor user can easily define howinformation should be presented. It should be noted that, while theinformation described below is with respect to an editor user creatingan information presentation, those of ordinary skill in the art willrecognize that a user can also use system 100 to view/playback the in1brmation presentation. In addition, it should be noted that, in oneembodiment, the system 100 can be used exclusively to define or playback/interact with the information presentation. While in other,embodiments, the system 100 can be used to do both.

FIG. 21 illustrates the multi-track timeline area approach in detail,according to one embodiment. FIG. 21 shows a timeline and timeline areawith multiple tracks 2105. The tracks can comprise asset tracks 2110 orcontrol tracks 2115. Other tracks are also possible. For example,referring to the legend in FIG. 19, in 2120 of FIG. 21, when, in editormode, the user can interact with the asset layout control track andpossible subsequent additional user interface prompts and interactionsto define a state change to occur to the information presentation at agiven point in time, in particular, defining a particular layoutpertaining, to assets to be executed at said point in time. In the sameway of interacting and defining a change of state of the informationpresentation, however through an interaction with the user layoutconstraint control track, the editor user can set/change the constraintsor capabilities of the user to modify the layout of the assets in anyway, whereby the user layout constraint can display a lock or variousother graphical indicators to represent the particular state of theinformation presentation in regards to the constraints set to the layoutat said point in time. For example, in 2120, a particular layout for agiven point in time can be defined and displayed on the asset layoutcontrol track and a particular set of layout constraints for aparticular point in time can be defined and displayed on the user layoutconstraint track. The layout icon on the asset layout control track canillustrate the layout at the given point in time and the lock icon canillustrate constraints on the user's ability to control the layout atsaid point in time. In 2125, a state change indicator pin can indicatethat there is a change in state for a given point in time in theinformation presentation and can be “draggable” or movable by the userusing an input device to collectively adjust the time for all statechanges for that given point in time of the information presentation. In2130, a change in state of the information presentation, for example theslide number to be displayed of the slide show presentation asset withinthe information presentation can be indicated. In 2135, an asteriskindicates that a pop-up external reference (e.g., wiki widget, glossarywidget) will be activated and displayed. In 2140 an assessment asset canbe activated and displayed. The amount of time that has a solid dark boxaround it can indicate the time for the assessment to be completed. In2145 a calculator widget can be activated and displayed. 2150illustrates a layout change indicator. 2155 illustrates apreview/playback indicator, which can represent the current time whilethe information presentation is being played back in order to bepreviewed by the editor user. In 2160, an activity within theinformation presentation can be activated using the activity widget. Inthis example, the activity that is activated uses a spreadsheet widgetand calculator widget. The green circle with white infinity symbol canindicate that there is no time minimum or limit for this activity.

FIG. 22 illustrates a number of interactions that a user can make with amulti-track timeline embodiment and how the context of a userinteraction can be used by the system 100. For example, the user inputanalyzer and router module 240 can be used to implement the contextualand predictive user interface capabilities of the system and within theuser interface. This can attempt to predict an editor's desires based onwhere the user clicks. Clicks on different parts of the timeline area,anywhere on the stage, or anywhere on the user interface, and related toany point in time in the information presentation can trigger differentresults, offering users a different set of options each time.Effectively, the system and application can attempt to anticipate aneditor's actions and desires based on the context of their click (e.g.,using pop-ups with choices for actions as described in more detailbelow). FIG. 22 and FIG. 33 illustrate how the contextual and predictiveuser interface capabilities of the system might be implemented whereinthe application is configured such that user interactions with thetimeline area trigger such capabilities. It is important to note thatsuch capabilities can also be implemented by and triggered with userinteractions with the stage, or anywhere on the user interface, andrelated to any point in time in the information presentation.

FIG. 33 illustrates a method by which the user input analyzer and routermodule 240 can implement the contextual and predictive user interfacecapabilities, according to one embodiment. In FIG. 33, in 3305, a usergestures (e.g., interacts, clicks, etc) on the timeline area. In 3310,user input analyzer and router module 240 determines which asset ortrack was closest to or received the gesture. For example, referring tothe multi-track map on FIG. 22, the user input analyzer and routermodule 240 can determine which of the following assets or tracks wereclosest to or received the gesture: the time source asset track, ageneral asset track, a real time interaction control track, an assetlayout control track, the application layout control track, the userlayout constraint track, and any assets that are present on any of thetracks. In 3315, the potential intended actions can be determined basedon the context of the gesture, where the context of the gesture used indetermining the potential actions includes which asset or track wasdetermined to be gestured to as well as the state of any of the variousassets or tracks or information presentation at any given point in time.In 3320, potential actions can be presented to the user (e.g., in apop-up screen or box) with the most likely intended action displayedfirst.

FIG. 34 illustrates a multi-track embodiment. In this illustration, avideo document asset, a slide show asset, an audio/video chat widget(with 3 users displayed), a calculator widget, and a text chat widgetare displayed on the stage as defined in the information presentation.In addition the multi-track timeline is displayed and can allow forinteraction and/or defining of the state of the information presentationover time. In this illustration, each asset is defined to be displayedand used for the entire duration of the information presentation, asillustrated on the timeline and the various tracks. From top to bottom,tracks 1-4 are general asset tracks and display: 1) the video asset; 2)the slide show asset, displaying slide #1 for the duration of thepresentation; 3) the calculator widget; and 4) the text chat widget. Inthis embodiment, the audio/video chat widget is displayed on thereal-time interaction control track.

Example Embodiments

FIGS. 23-32 illustrate various embodiments of the system 100. FIGS.23-26 illustrate an information presentation as played back by a userusing a runtime mode embodiment (referred to below as a player), as wellas the same information presentation as defined by an editor, accordingto one embodiment. In these figures, both the information presentationand timeline area shown to the user, as well as the corresponding editoruser set-up and timeline area are shown, illustrated together. In FIG.23, a timeline indicating the detailed structure and changes of stateover time of an information presentation is illustrated. In 1, theinformation presentation can start with a video asset presented insingle asset layout mode, with the user being locked to this layout modeonly; not able to change the layout in any way, and potentially, asdefined by the editor, not able to add or remove other assets to theinformation presentation. In 2, when the time controller moduledetermines that the elapsed time has been reached, it will execute thedefined state changes, which are signified symbolically and graphicallyin the timeline area user interface, wherein the state changes are: 1)the addition of a new asset, in particular a slide show asset 2) thedisplayed slide show slide from the slide show asset being slide number1, and 3) a change in layout of the information presentation,specifically a two asset layout mode where the slide show and video arcnow displayed together, side by side. FIGS. 23, 3-4, and FIGS. 24-27correspond and operate within the system accordingly.

In FIG. 23, in 3, the timeline area and/or track display a number “2”signifying the displayed slide show asset slide from the slide showasset being changed to slide number 2. (Note that since there is nostate change indicted on the timeline, no layout state change occurs atthis point in time.) In 4, the asterisk on the timeline can indicate theactivation and display of an external resource widget (e.g., externalweb page, external web page link) where additional information may befound.

In FIG. 24, a timeline indicating a pan of an information presentationis indicated. In 1, a player can reach the time in an informationpresentation whereby a state change(s) is to occur, wherein the statechange(s) to occur is for the assessment widget to be activated. Oncethe player (i.e., creation/playback application) reaches the applicablepoint in time, the video asset can be paused, and the assessment can beactivated for the user to complete. This assessment can be timed, asrepresented by the clock icon. In 2, once the assessment is finished,the information presentation can continue. In 3, the player andinformation presentation reaches another set of state changes within theinformation presentation: a calculator widget is added to the stage, thestage switches from a two asset layout to a three asset layout, and, assignified by the unlock icon in the user layout constraint controltrack, the state of the information presentation and application is nowsuch that a user can change the layout of the assets without constraint.

In FIG. 25 a timeline indicating another part of an informationpresentation is indicated. In 1, an audio asset can be triggered to play(and in this example to replace the video that was previously playing).Because the audio asset docs not require a visual, the informationpresentation can switch from a three asset layout to a two asset layout,as indicated on the asset layout control track. In 2, another statechange indicator can indicate another change in the informationpresentation. In this example, the player approaches another statechange (visually represented by a pin), and the slide show can switch toa document. The layout can remain a two asset layout. The number “22”can indicate that page 22 should he the part of the document presented.

In FIG. 26, a timeline indicating another part of an informationpresentation is indicated. In 1, a player can reach a set of statechanges that represent a do timed interactive exercise (indicated by theinfinity sign on the timeline) that uses a spreadsheet widget and acalculator widget. As defined by the editor user, the informationpresentation changes state to display a spreadsheet widget and acalculator widget on the screen and in a two asset layout. In 2, oncethe user is finished with the activity and prompts to continue, theinformation presentation continues, with the creation playbackapplication 120 immediately executing a set of state changes. In thisinstance, the state changes are such that the spreadsheet widget isremoved from the stage and a four asset layout is applied, containingthe calculator widget, a video asset, a document viewer widget, and achat widget.

FIGS. 27-28 illustrate automatic and manual modes according to severalembodiments. FIG. 27 illustrates an information presentation as definedby an editor user in a multi-track embodiment in editor mode as playedback by a user in a runtime mode embodiment in automatic mode, 2705,illustrates the state of the information presentation as defined by aneditor user. In 2710, the editor user has defined the state of theinformation presentation at a given point in time to be such that acalculator widget, a video asset and a slide show editing/viewing widgetare to be displayed in a three asset layout. When the given point intime is reached within the information presentation, if in automaticmode, said state changes are executed, leaving the informationpresentation in a state as exactly defined by the editor user. For thatpoint in time. In 2715, the editor's configuration dictates that thelayout automatically reconfigures the information presentation from athree asset layout to a two asset layout, displaying the calculatorwidget and the slide show editing/viewing widget, with an audio assetalso playing.

FIG. 28 illustrates an information presentation as defined by an editoruser in a multi-track embodiment in editor mode as played back by a userin a runtime mode embodiment in manual mode. As shown and described in2805, an editor user can define constraints related to a user's abilityto modify the information presentation during playback. The constraintscan be related to modifications such as, but not limited to: adding orremoving assets, resizing assets, positioning or moving assets,modifying assets, or interacting with assets, or any combinationthereof. For example, a lock icon can indicate that certain thingscannot be changed. As per allowed or disallowed by the constraints, auser can perform/invoke such modifications to the informationpresentation and can cause the creation playback application 120 to bechanged from automatic mode to manual mode. In 2810, the editor user hasdefined the state of the information presentation at a given point intime to be such that a calculator widget, a video asset and a slide showediting/viewing widget are displayed in a three asset layout. In thisexample, the player is in automatic mode and reaches such point in timeand executes such changes as defined by the editor. The user thendecides that he/she does not need the calculator and can thereforeremove the calculator widget, which results in the restoration to thetwo asset layout. The removal of a widget can be done by having a userhighlight an icon representing the asset, and then pressing delete. In2815, it is shown how some state changes arc not executed because theuser is in manual mode. Those of ordinary skill in the art will sec thatother methods are also possible for deleting an asset.

The user can also select whatever layout the user prefers using avariety of means. By means of a user modifying the informationpresentation during playback, such as selecting/configuring/changing thelayout or adding/removing assets from the screen, or inviting otherusers to collaborate in collaborate mode, the creation/playbackapplication 120 can change from playing the information presentation inits predefined formal (e.g., automatic play mode) into manual play mode,wherein the creation/playback application in manual play mode can nolonger implement some or all the predefined information presentation'sstate changes, such as the display/activation of assets and layoutchanges. The user can immediately switch back to automatic mode with asingle user gesture, such as clicking on a button. In the example, afterthe user removes the calculator widget, the creation/playbackapplication 120 is now in manual mode. As the system in run-time modecontinues to play the information presentation, and the time and timeline of the information presentation reaches the respective point intime, the state of the information presentation as per defined by theeditor user during the use of the system in editor mode is such that theinformation presentation uses a three asset layout, displaying the videowidget, the slide show editing/viewing widget with loaded slidepresentation on slide number 9, and the calculator widget. However, someor all of these state changes, may not be executed, as a result of thevarious constraints that may have been implemented as part of theinformation presentation, and because the creation/playback application120 is in manual mode. The state change where an audio asset is playedis executed, while the video asset (which video has ended) remains onthe screen in the layout defined by the user. In addition, the user cancontinue to modify the information presentation as described above. Forexample, an asset can be added by having the user click on an iconrepresenting the asset browser, and then drag and drop different assetsto a certain pan of the screen. Those of ordinary skill in the art willsec that other methods arc also possible for adding an asset.

FIGS. 29A-31 illustrate blank timeline click, defined time asset clickand slide number gesture and scenarios, as can be implemented by theuser input analyzer and router module 240, providing contextual andpredictive user interface capabilities, according to various multi-trackembodiments. These figures illustrate a small subset of the variousembodiments the user interface module 235 can utilize to presentpredictive and contextual user interaction capabilities. In FIGS. 29Aand 29B, a sequence using a wizard or dialogue box approach can beutilized, according to one embodiment. In 1, a single asset layout box avideo on the timeline can be shown. The editor can click on a blank areaof the timeline, such as a blank area on a general widget track. In 2,pop-up menus can present the editor user with a list of options. Theseoptions can be dynamic and ranked by what an editor's highest probabledesire is, based on where the editor clicked on the timeline. In 2, theeditor user can add a new asset at the specified time, wherein the timeis also determined based on where the editor user clicked. In 3, basedon the editor's previous selection of adding a new asset, a new optionsmenu can he presented asking the user to select what type of asset hewould like to add to the timeline and track. In this example a slidedocument asset can be added. The editor can then be sent down a uniquepath of menus based on the type of asset he selected to add. Forexample, in 4, the editor user can next choose a slide document toupload. In 5, the editor user is asked which slide he would likedisplayed at the given or determined point in time. A particular slidecan be selected. In 6, because this is a newly added asset, anotification is shown that this addition will change the layout. Adefault layout can be automatically selected, and the editor user can begiven the option to choose an alternative layout.

In a manner similar to FIGS. 29A and 29B, FIGS. 30A and 30B illustrate aslide number click in a multi-track embodiment. In 1, an editor canclick on an existing slide number. In 2, the editor's slide selectioncan be displayed, along with summary information, a list of possibleactions, etc. In this example, a new slide is being inserted. In 3, alist of available slides can be displayed. Optionally, the editor mayload slides from a new file. In 4, the editor can drop a new slide ontothe timeline, represented by a pin with a slide number in it. In 5, thenew stage layout can be shown. (Note that in this example there was nostage change as the new slide was inserted on an existing slide track.)

In a manner similar to FIGS. 29A and 29B, FIGS. 31A and 31B illustrate adefined time asset click in a multi-track embodiment. In 1, a two assetlayout is displayed. An editor can click on the track/video asset attime 12:22. In 2, a context aware pop-up menu can be displayed with alist of options. In 3, based on the user's previous selection, a popuppresenting the user a list of types of time control/defined time assetscan be presented. In 4, based on what type of asset a user selects toadd, various files can be shown for selecting or uploading. In thisexample, an ‘Audio Asset’ has been selected. In 5, an audiorecording/playback widget can be loaded into another subsequent popupmenu, allowing the editor to preview the audio file/asset. In 6, theeditor can be notified that there will be a change in the layout becausethe audio file has nothing to display. In 7, the user has completed theaddition of the asset (i.e., audio file) and the change in layout isshown within the application and system.

FIG. 32 illustrates various information presentation layouts, accordingto several embodiments. FIG. 32 also illustrates various timeline andcontrol configurations.

FIGS. 35-36, 39 and 41-47 depict several embodiments of the system 100according to various usage scenarios within the context of the one ormore information presentations.

FIG. 35 illustrates editor users interacting with the informationpresentation (e.g., defining/controlling the information presentation),and users interacting with each other in both real-time and nonreal-time.

FIG. 36 illustrates editor users interacting with the informationpresentation (e.g., defining/controlling the information presentation),and users interacting with each other in both real-time and nonreal-time, whereby the system 100 is in use in a live setting. In thisillustration, the editor user can give an in-person presentation to alive audience. In place of, or in conjunction with control orpresentation instruments (e.g., including, but not limited to: acomputer, laptop, mobile device, tablet computer, television,video/teleconferencing system, interactive whiteboard, or otherclassroom or presentation device and/or tool; or any combinationthereof) a presenter can use real-time, in a live setting, an editormode embodiment of the creation/playback application 120 to present andcontrol an information presentation using assets with and for a live(i.e., in person) audience, as well as with and for other users who areviewing and interacting with the information presentation remotely usingthe system. For example, the editor user could use an embodiment of thecreation/playback application 120 in editor mode either in place of, orto control, or in conjunction with a traditional instrument, such as acomputer using presentation software and presentation slides (e.g.,PowerPoint, Apple KeyNote, etc.) or document software (e.g., MicrosoftOffice, Microsoft Word, Adobe Acrobat, etc.), wherein thesoftware/slides/documents are traditionally used by being projected in alarge format for the in-person live audience to see. The use of thecreation/playback application 120 in conjunction with such traditionalinstruments could be implemented using several components of thecreation/playback application 120, as well as by using special softwareplug-ins or hardware devices connected or integrated with suchinstruments. In addition, the editor user can set the editor modeembodiment into the record mode and record his/her real-time editing anddefining of the information presentation, to be: (1) immediatelyavailable to he replayed as a predefined, on-demand informationpresentation; or (2) further edited, then replayed as a predefined,on-demand information presentation; or any combination thereof. Itshould be noted that, in FIG. 36, both in-person live audience users aswell as remote users can be using an embodiment of the creation/playbackapplication 120.

FIGS. 41A and 41B illustrate multiple usage scenarios of thecreation/playback application 120, specifically where there is apredefined information presentation and configuration of real-timewidgets, according to one embodiment. In this case, the editor userwants users of the information presentation to be notified when he isonline and available for interaction and collaboration with users. Inaddition, the editor user wants to encourage interaction between theusers. The editor user can implement and configure the use of both areal-time chat widget and a real-time audio/video chat widget so as tobe available to collaborate with the users using such widgets.

For example, in Scenario 1, a user Hiroko can view Professor X'spredefined information presentation of a slide show in 1. In 2, Hirokocan view Professor X's predefined information presentation of a slideshow and video. In 3, although Professor X is not online, Hiroko canactivate the chat widget and video-chat widget and collaborate andinteract with other available friends and information presentationviewers in real-time. When Hiroko modifies/edits the run-timeinformation presentation and adds the chat widget, the informationpresentation mode can change to manual mode.

In Scenario 2 of FIG. 41B, in 1, John, like Hiroko, can view ProfessorX's predefined presentation of a slide show. In 1(b), Professor X cancome online. This can cause the predefined layout to be interrupted toexpose the chat widget. In 2 and 3, even though the chat widget isappearing, because it was pre-configured to be able to appear, theinformation presentation is still in automatic play mode.

In Scenario 3 of FIG. 41B, in 1, Julie, like Hiroko and John, is a userviewing Professor X's predefined presentation. In 1(b), during Julie'sviewing session. Professor X's office hours occur. This can interruptthe predefined layout to expose the chat widget. In 2 and 3, a slideshow and video appear along with the chat widget. Because the chatwidget was pre-configured to be able to appear, the informationpresentation is still in automatic mode.

FIG. 42A-42C illustrate a usage scenario of the creation/playbackapplication 120, specifically where there is a predefined informationpresentation in run-time mode with real-time widgets and real-timecollaboration, according to one embodiment. In 1, a user Julie iswatching a predefined information presentation and using the documentediting widget to create/edit notes as she views the informationpresentation. In 2, Julie then activates the user list widget toidentify other users who arc available so that Julie can discuss and/orcollaborate with on a topic she is confused about. In 3, Julie secs thatshe has some “friends” available and online, as well as other usersviewing the information presentation that is available. Julie pauses theinformation presentation and activates the chat widget to seek out helpfrom someone. Julie had the information presentation in automatic mode,but by activating the chat widget and thereby adding it to the screen,the information presentation is taken out of automatic play mode and putinto manual play mode. In 4, the chat widget is automatically laid outon the stage. Using the chat widget, Julie can ask her friends,classmates and other users for some clarification about the topics. In5, Julie engages one of the available users. John (who is a friend andclassmate), in real-time, first over chat using the chat widget. Julieasks John to view and collaborate within the information presentation,and though he could do so from his mobile phone, John asks Julie to waituntil he can access the information presentation from his laptopcomputer. In 6, Julie activates collaboration mode so that John can beengaged within the system. In 7, when John arrives home, he opens up hise-mail and clicks on the link Julie sent him using the system 100. Helogs into the system and is taken to the information presentation.John's information presentation and application instantly becomes syncedwith Julie's information presentation as they start the collaborate modesession. The system can automatically activate the audio/video chatwidget instead of the text chat widget. All collaborators in theinformation presentation can have their own cursor and can together atthe same time vim every other collaborator's cursor, as well as eachbeing independently able to control the information presentation and seeevery single interaction or modification a user makes to the informationpresentation, Julie then shows John the information presentation segmentthat she has trouble with and discusses it via audio and video chat withJohn. In 8, Julie is still having difficulty understanding what John isexplaining, so John decides to share his notes with Julie. John thentakes control of the mouse and activates a second document editingwidget and then loads his notes into the widget. In 9, a warning isdisplayed notifying John that he is about to share his notes. Johnacknowledges that he wishes to continue and selects that he would liketo enable collaborative editing. in 10, the information presentation andstage automatically reconfigure to show both John and Julie's notes onthe screen, while relegating their video chats to the edge of thescreen.

FIGS. 43A and 43B illustrate another usage scenario of thecreation/playback application 120, where there is also a predefinedinformation presentation in run-time mode with real-time widgets andreal-time collaboration, according to one embodiment. FIGS. 43A and 43Billustrate how Julie and John can engage in real-time use of the systemwith Professor X. Within the collaboration mode session, the three usersmake use of various real-time widgets and collaborative features,including the calculator widget, the whiteboard widget, the documentediting widget, synced views of the information presentation, and theability to view every other participant's cursor,

FIG. 44 illustrates another usage scenario of the creation/playbackapplication 120, where there is an editor user in a live setting inrecord mode, according to one embodiment. The editor user can he infront of live audience and can interact with various other users of theinformation presentation, including live and real-time users,non-real-time users, and on-demand users. Scene I depicts a presenterdelivering a live presentation in New York at 12:00 PM. As the presenterdelivers the in-person presentation, he uses editor mode to edit/definethe information presentation “on the fly” (e.g., changing the state ofthe information presentation, such as adding/removing assets andchanging layout, etc.) throughout the duration of the informationpresentation. Additionally, the system is set into a record mode andrecords the state changes and their times as the informationpresentation is being edited/defined and delivered in real time.

Scenes 2 and 3 take place in London where it is 5:00 PM. Using thecreation/playback application 120 in run-time mode, users arc able towitness and interact with the information presentation and with otheravailable users, including the editor user, in real-time, using suchcapabilities as the chat widget, video chat, etc. In Scene 2, the useris viewing the information presentation in automatic mode. In Scene 2,the presenter has specified a 3 asset layout that includes a live videofeed, a slide show presentation, and a calculator widget. In Scene 3,however, the user has opted to run the live information presentation inmanual mode, configuring his workspace (as allowed by the editor user)to his liking. The user in Scene 3 has opted to listen to the audiostream of the live information presentation, while viewing the slides,and activating the chat widget to converse with other users.

Scenes 4 and 5 take place in Tokyo, Japan. Since it is 1:00 AM in Japanat the time of the live information presentation, users will opt not tonot sec the information presentation until they wake up the nextmorning. In Scene 4, a user is viewing a published on-demand version ofthe information presentation at a later point in time (non-real time).The user can view it in automatic mode, seeing the informationpresentation exactly as the instructor intended/defined/recorded, or theuser may (where permitted by the editor) modify the layout of theinformation presentation and view it in manual mode. In Scene 5, a useris viewing a non-real time version of the information presentation, butis chatting with the professor who delivered the presentation at a timein which both parties happen to be online.

FIGS. 45A and 45B illustrate another usage scenario of thecreation/playback application 120, where there is also an editor user ina live setting in record mode, according to one embodiment. In Scenario1, an editor user pre-configures and predefines some aspects of aninformation presentation. In 1, Professor X loads a number of slide showpresentations he wishes to use during his live presentation into thesystem 100 using the creation/playback application 120. In 2, ProfessorX configures the information presentation so that users viewing it inautomatic mode will view a video that will be taken of him. Theinformation presentation will show the slide show presentation on theleft and the video on the right. In 3, Professor X decides to allowusers the ability to chat during the information presentation, so usingthe creation/playback application 120, he configures the chat widget torun continuously and to be available for other users of the system 100who are viewing the information presentation. Because he knows that someusers may not see the information presentation until the next day, hewants the system to notify them whenever he is online, so they canfollow up with him and ask questions directly, and uses thecreation/playback application 120 to configure such behavior.

In Scenario 2, the editor user loads, then uses the predefinedinformation presentation within the creation/playback application 120 ineditor mode, in front of a live audience setting during a presentation.In addition, he can have set the creation playback application 120 intorecord submode. In 1, Professor X interrupts his informationpresentation using the creation playback application 120 and thepredefined information presentation to show the audience how tocalculate something. In 2, Professor X activates the calculator widgetand begins showing the audience how to calculate the answer to aproblem. He asks the audience to try to calculate it themselves anddiscuss their results. In 3, once complete, Professor X decides to takequestions from students and other users of the system 100 viewing theinformation presentation and activates the chat widget that is availableand running in the background to be displayed for both live audiencemembers and all other users of the system 100 viewing the informationpresentation.

In Scenario 3, a user accesses the system and the informationpresentation from a remote location, in real-time as it is being definedand delivered by the editor user. This user uses the system in automaticmode, such that the information presentation is played back exactly asit is being defined by the editor user. Thus, in 1, John is in Londonand watches Professor X's information presentation live in real time. Hesees a video of Professor X on the right and the slides on the left. In2, since John is viewing the information presentation in automatic mode,when Professor X activates the calculator widget, John's screenautomatically reorients itself to display the calculator widget. In 3,once Professor X activates the chat widget, John's screen reorientsitself to display the chat widget so that he can have discussions withusers.

In Scenario 4, another user also accesses the system and the informationpresentation from a remote location, in real-time as if is being defineand delivered by the editor user. However, this user opts to inactivateand hide some of the widgets that the presenter/editor user isdisplaying, thereby putting the application into manual mode. If a slatechange was to be activated but cannot due to the inactivation of some ofthe widgets, the user is notified of the specific state change that wasto be implemented, but was not implemented due to being in manual mode,through a notification indicator popping up on the screen. In thisscenario, the editor user added a calculator widget to display withinthe information presentation, and the user was notified of this statechange via a pop-up window. Thus, in 1, Julie, like John views theinformation presentation from a remote location in real time. She findsthe video footage too distracting and so opts to only see the slides byclosing the video widget. In 2, because Julie is not in manual mode,when Professor X adds a calculator to the screen, she only receives apopup notification indicating that a state change has occurred. If shedesires, she can click on the pop-up to see the predefined stage as itappears in automatic mode. In 3, when Professor X activates the chatwidget, Julie remembers a question. She clicks to activate the chatwidget and joins the discussion Professor X has started.

In Scenario 5, the user watches the information presentation, initiallyin automatic mode, as it was recorded/defined by the editor user hoursearlier. The user then changes and customizes the layout, therebychanging to manual mode, and then uses the real time interactive textchat widget. Thus, in 1, Hiroko is in Tokyo during the class, so sheviews the information presentation in non-real-time the next day. In 2,the screen can be configured to Hiroko's liking. For example, whenProfessor X activated the calculator widget. Hiroko can choose to closethe video widget. In 3, because Professor X activated the chat widgetand enabled it to run continuously, Hiroko can chat with other presentviewers of the information presentation or other users of the system.Moreover, because Professor X decided to make himself “known” when helogs on. Hiroko can ask him questions in real-time.

FIGS. 39A and 39B illustrate another usage scenario of thecreation/playback application 120, where an existing informationpresentation's data can be edited, according to several embodiments. Asindicated in 1, an information presentation could have been deliveredusing an editor mode embodiment in record mode one of two ways. In 1a,the editor user (e.g., a Professor) could have used the application anddelivered an information presentation both in front of a live audienceand to other users of the system comprising an online audience inreal-time. In 1b, the editor user could have delivered an informationpresentation to only other users of the system that comprise an onlineaudience, similar to a “live webinar” or web conference. In 2, theeditor user loads an information presentation, then previews theinformation presentation, locates an area he wants to change, so hepauses the information presentation. In 3, the editor user switches fromruntime/playback mode to edit mode by right-clicking on the time lineand selecting edit. In 4, a multi track time line appears on the screenand the user is in edit mode. In 5, the editor user clicks on therelevant slide display indicator on the multiple track time line anduses the user interface to switch it out with an updated version of theslide. In 6, the editor user replaces a portion of the video that wasalready in place in the information presentation with an audio file.This can be done by clicking on the time line asset video at thespecified time, uploading his new audio file, and locking the stage intoa single asset view so viewers only hear his voice and see the newlydefined slide. In 7, once the editor user is finished making edits, hecan publish the information presentation for on demand viewing. In 8,the information presentation can be put on the setting “publish toserver for viewing”). In 9, once the editor user publishes theinformation presentation to the server for viewing and configures thepermission settings, the information presentation is ready to be viewedonline.

FIGS. 46A-47 illustrate another usage scenario of the creation/playbackapplication 120, where collaborative creation and editing of aninformation takes place, according to one embodiment. In 1, John, Julieand Hiroko have an assignment to build an information presentation as ateam. Using the system and the creation/playback application, Johncreates a new information presentation. John then invites Hiroko andJulie to the workspace so they can work together. In 2, once all threemembers arc in the workspace, they begin collaborating via video chat onhow they want the information presentation to go. They decide to startwith a video introduction by Julie. In 3, Julie opens her tools menu anddrags the video widget on the screen. In 4, a pop-up menu of optionsappears on Julies screen and Julie chooses to record a new video segmentwith her web cam. In 5, because Julie is recording a web cam video in amulti-user environment, she switches her environment to private mode,which allows her to isolate herself from other state changes that usersare doing, so she can focus on the task at hand. In private mode, shewill receive small/subtle notifications of ongoing changes in theinformation presentation in the corner of the screen, so she has a logof events that are occurring and can receive feedback from herteammates. In 6, all three users can see Julie deliver the web camintroduction, as she is recording. While Julie is delivering thepresentation, Hiroko becomes inspired to create the accompanying slidesfor Julie's introduction and adds the editing widget to the stage byclicking an a blank track. In 7, the slide show editing widget appearson the stage next to the video segment that Julie is currentlyrecording. John is able to see Julie recording her presentation andHiroko building/editing the accompanying slide show using the slide showediting widget simultaneously, in real-time. In 8, when Julie and Hirokoare finished, John takes charge and begins working on the timing of theslide transitions, while Julie and Hiroko provide verbal guidance.

Continuing on in FIG. 47, in 9, after John is finished syncing Hiroko'sslides to Julie's presentation, they collaboratively review thepresentation and decide what additional tools, like the calculatorwidget, to add to the information presentation. In 10, as Hiroko andJohn are doing this, Julie feels that it might be a good idea to starttaking notes on potential questions they might want to ask viewersthrough the presentation. She activates her note pad (not her works acelike she did previously) in private mode so as not to disturb John's nodHireiko's work, and begins taking notes. In 11, once John, Hiroko, andJulie have finished configuring the widgets on the stage. Julie sharesher note pad with the team, showing them all the questions she hasjotted down. They collaboratively edit the questions on Julie's notepad. Once finished, Julie emails a copy to everyone, so they have theirown copy of the content. In 12, one they have finished building the itpresentation, they preview the information presentation in its entiretyin automatic mode. In 13, now that they have reviewed the informationpresentation and are satisfied with it, John configures the settings forthe information presentation before publishing it to the web. In 14, theinformation presentation settings are shown.

FIGS. 34 and 49-67 are screen shots illustrating various single trackand multi-track embodiments of the creation/playback application 120,and various information presentations and assets. With respect to thesefigures, and other examples screens illustrated throughout thisdisclosure, those of ordinary skill in the art will understand that allof these examples screens can be sequenced together in different ways tocreate and playback many different types of activities using theactivities widget.

FIG. 49 illustrates a single track embodiment with a document viewerwidget displaying a document, a video asset and a document editingwidget displayed.

FIG. 50 illustrates a single track embodiment with a video asset, aslide show editing/viewing widget displaying a slide document, an assetbrowser widget and a chat widget (labeled class discussion) displayed.

FIG. 51 illustrates a single track embodiment with an assessment widgetdisplaying a quiz modally within the information presentation and thecreation/playback application.

FIG. 52 illustrates a single track embodiment with a video asset, aslide show editing/viewing widget displaying a slide document, and abookmark widget displayed.

FIG. 53 illustrates a single track embodiment with a spreadsheet widgetand a calculator widget displayed modally within and using an activitywidget within the information presentation in the creation/playbackapplication.

FIG. 54 illustrates a single track embodiment displaying: a documentviewer widget displaying a document, a video asset, and a spreadsheetwidget.

FIG. 55 illustrates a single track embodiment displaying: a video asset,a spreadsheet widget, and a calculator widget.

FIG. 56 illustrates a single track embodiment displaying: a documentviewer widget displaying a document, a video asset, and a calculatorwidget.

FIG. 57 illustrates a single track embodiment displaying: a documentviewer widget displaying a document, and a calculator widget.

FIG. 58 illustrates a multi-track embodiment displaying: a video asset,a slide show editing/viewing widget, a calculator widget, and a documentediting widget.

FIG. 59 illustrates a multi-track embodiment displaying: a video asset,a document viewer widget displaying a document, and a calculator widget.

FIG. 60 illustrates a multi-track embodiment with a video asset anddocument viewer widget displayed.

FIG. 61 illustrates a multi-track embodiment with a video asset, acalculator widget, and a document editing widget displayed.

FIG. 62 illustrates a multi-track embodiment with a video asset and acalculator widget displayed.

FIG. 63 illustrates a multi-track embodiment displaying: a video asset,and a slide show editing/viewing widget displaying a slide documentdisplayed.

FIG. 64 illustrates a single-track embodiment displaying: a video asset,a slide show editing/viewing widget displaying a slide document, a chatwidget, and a user list widget displayed as a popup (labeled “Chat”).

FIG. 65 illustrates a single-track embodiment displaying: a video asset,a slide show editing/viewing widget displaying a slide document, a videochat widget (with three users shown), and a discussion widget (labeled“Topic: Accounting”).

FIG. 66 illustrates a single-track embodiment displaying: a video asset,a slide, show editing/viewing widget displaying a slide document, avideo chat widget (with 8 users shown), and a discussion widget (labeled“Topic: Accounting”).

FIG. 67 illustrates a single-track embodiment displaying: a video asset,a slide show editing/viewing widget displaying a slide document, a videochat widget (with four users shown), a text chat widget, and a user listwidget displayed as a popup (labeled “Chat”).

FIGS. 68-74 arc screen shots illustrating various single track andmulti-track embodiments of the creation/playback application 120. FIGS.68-74 illustrate single-track embodiments, and sequentially togetherdemonstrate several capabilities of the system and creation/playbackapplication, including the use of the discussion board widget inimplementing an “assignment”.

FIG. 68 displays: a video asset, and a slide show editing/viewing widgetdisplaying a slide document.

FIG. 69 illustrates a speaker/presenter within the video assetinstructing users orally as to art upcoming activity which the user wailhave to participate in. The video asset illustrated could be a live andreal-time video stream asset, or a prerecorded video asset.

FIG. 70 illustrates a modal dialog box introducing the activity to usersof the system, as well as providing instructions regarding the activityto users of the system. In FIG. 70, the user is able to click on thegreen “Go” button which results in the creation/playback applicationchanging to the state depicted in FIG. 71, which continues the sequence.Alternatively, an editor user using the system to deliver theinformation presentation in a live in real-time, may be controlling thestate of the presentation such that such editor user controls theadvancing of the creation/playback application for all users to thestate depicted in FIG. 71.

FIG. 71 illustrates a rich text entry component (which may be a documentediting widget) for users to enter responses to be posted to adiscussion board widget. In addition, FIG. 71 illustrates an assetbrowser widget, which is depicted as a pop-up list of files, appearingin response to a user's clicking on the button depicted (labeled“Assignment Resources”). The user may enter their response into/usingthe rich text entry component. The system may receive and save suchresponse to the associated discussion board widget, such discussionboard widget being depicted in FIG. 72.

The discussion board widget depicted in FIG. 72 shows many differentdiscussions, each of which may be submissions from users using the richtext entry method described and depicted in FIG. 71.

FIG. 73 continues the sequence, depleting what any user of the systemmight see upon selecting, to view a discussion from the list depictedFIG. 72.

FIG. 73 depicts a threaded format discussion within the discussionwidget.

FIG. 74 illustrates: a single-track embodiment with a document viewingwidget editing/viewing widget displaying a slide document, a video chatwidget (with 3 users shown), and a discussion widget (labeled “Topic:Accounting”).

FIGS. 75A-75E together further illustrate many capabilities oldie system100 and creation/playback application 120. Specifically, these figuresillustrate the creation of a predefined information presentation using amulti-track editor mode embodiment. It is important to note that thesame sequence of events described with respect to FIGS. 75A and 75B, aswell as with respect to all other scenarios described in this disclosurecould be used in scenarios with live viewers/users/participants and withor without the multi-track display being used.

For example, referring to FIGS. 75A-75E, in 1, Professor X may becreating a predefined information presentation using an editor modeembodiment using his desktop computer, tablet PC, mobile/smart phone, orother computing/controller device, or any combination thereof. In 2, thecreation playback application opens in editor mode with the multi-tracktimeline visible. Professor X sets the initial state of the presentationto use a video recording/playback widget and a document viewing/editingwidget. In 3, Professor X activates record mode. The user interfacepresents various options based on the context of the user interactionand configuration of the information presentation at the given point intime. In 4, Professor X clicks “Start Recording” and proceeds to recordhimself giving a presentation. At this point, Professor X is recordinghimself on video and audio and proceeds through the first few pages ofthe document. In 5, a short way into his presentation, Professor Xdecides he'd like to better explain a concept and will engage a fewother assets and widgets to do so. First, using the asset browserwidget, Professor X locates a relevant diagram. It is important to notethat Professor X could have also chosen to create and display a newdiagram. Referring to FIG. 75B, in 6, a detailed view of the multi-tracktimeline, demonstrating how during the recording of an informationpresentation, all changes of state of any sort to the assets, widgets orinformation presentation may be recorded, as well as displayed visuallyon the multi track timeline.

In 7, the user interface presents Professor X with various options foradding the diagram to the stage and information presentation using adiagram viewing editing/widget, including automatic layout options wherethe application automatically detects the best layout option. Theapplication can also automatically add the diagram asset or widget tothe stage without prompting the user, Professor X. In 8, Professor Xactivates the drawing widget. In 9, using various keyboard shortcuts,and/or on screen prompts, etc., Professor X is able draw on/withinand/or interact with both individual widgets or assets, as well as thestage as a whole. For example, Professor X makes some modifications tothe diagram in real time, the state changes for which are beingrecorded. Likewise, Professor X can draw shapes overlaying andoverlapping the various assets on the stage. It should be noted that 9is only one example of the combining of widget capabilities togetherwith other widgets, in this case the drawing suite widget being used andcombined with the diagram viewing/editing widget, as well as thedocument editing widget. In addition, 9 is also one example of theability to put assets on top of other assets, in this case, the drawingsuite widget being used to draw over top of the entire stage within thecreation/playback application 20.

Referring to FIG. 75C, in 10, Professor X decides he'd like to have somesupplementary information in the presentation, but leave the existingwidgets and layout intact. To do this, Professor X can use the layerswidget, to create a new layer on the stages.

In 11A, after creating the new layer, Professor X adds a new slide showediting/viewing widget to the layer on the stage. Professor X edits theslide show, adding some text in bullet point form, then adds acalculator to the screen and performs an example calculation, all ofwhich may still be recorded.

In 11B, Professor X then changes back to the first layer to continuewith the main portion of the presentation.

Referring to FIG. 75D, in 12, Professor X is done presenting thematerials and stops the recording process (e.g., by clicking a stopbutton). All of the changes of state of the information presentationhave been recorded and persisted by the system and creation playbackapplication.

In 13, all of the changes of state of the information presentation thathave been recorded are rendered and represented visually on themulti-track timeline. Professor X would now like to make some additionsand changes to the information presentation. Specifically, Professor Xwould like to make some areas of the diagram he used into links to: 1)one or more points in time in the information presentation; 2) otherassets/widgets; 3) other states of other assets/widgets; 4) other layersin the layers widget; 5) other information presentations; or 6) anyother URI or URL; or any combination thereof. Any of these options canbe displayed within the linked from/original information presentation oroutside of the information presentation. Those of ordinary skill in theart will see that there are other link types that may be created andapplied to objects in the system.

Referring back to 13 on FIG. 75D, Professor X may drag the timelineindicator/playhead across the timeline in order to quickly find thepoint in the presentation.

In 14, after locating the point in the it presentation where the diagramwas added, Professor X now begins interacting with the diagramviewing/editing widget and the application in order to create the linkshe would like.

In 15, it is demonstrated how, in this embodiment, by selecting andright clicking on an object (e.g., shape, image, text, etc.), ProfessorX can access the linking widget. In this example, Professor X is makingthe large blue arrow in the diagram link to another point in theinformation presentation, such that when a user clicks the arrow, theplayhead will jump to another point in time in the presentation and thepresentation will play from that point in time. The user interface inthis embodiment allows a user to enter a time in hours, minutes, andseconds format, or to use the timeline and playhead to select the pointin time (i.e., by clicking and dragging the playhead to the desiredlocation/time in the information presentation to link to).

Referring to FIG. 75F, in 16, Professor X continues to link severalother objects within the diagram, using all of the different linkingtypes. The creation of each link type works similarly to that describedin 15, wherein after right-clicking, then selecting a link type, theuser/editor (i.e., Professor X) is presented with special dialog boxesfor creating that link type. In the example in 16, Professor X islinking an object to another asset or widget. The menu presented allowshim to select an asset as well as some options for how the asset isdisplayed (e.g., add asset to stage, display asset modally). Those ofordinary skill in the art will see that there are many otherconfiguration options available for all of the link types.

In 17, Professor X finishes creating the presentation and uses the savecapability of the system 100 and creation/playback application to savethe presentation.

FIGS. 76A-76D together illustrate many capabilities of the system 100and creation/playback application 120. Specifically, these figuresillustrate the creation of a live delivery (and recording) of aninformation presentation using an editor mode embodiment. Referring toFIG. 76A, in 1 illustrates an editor user (Presenter X) in front of alive, in-person audience. Presenter X is using an editor mode embodimentof the system 100 and creation/playback application 120 which isintegrated with and connected to a number of control or presentationinstruments, specifically, for example: a tablet PC, multiple videocameras and/or a video/teleconferencing system, a touchscreen PC ordevice, interactive/touchscreen whiteboard, document camera(s) or otherspecialized cameras (e.g., surgical cameras), other audio/video or datasignal producing devices, and a projector. An audience is watching andparticipating (i.e., users) using the system 100 and creation/playbackapplication 120 and various runtime and editor mode embodiments. Theaudience is comprised of both live, in-person users, and remote users,all of which are able to view and participate using embodiments ondesktop PCs, tablet PCs, mobile/smart phones, and other computing andcontrol devices.

In 2, Presenter X is preparing to give a live presentation to both alive and remote audience. He will also be recording the presentation forlater editing or playback. Before or during the presentation. PresenterX can configure the application in many ways. Examples of such waysinclude, but are not limited to: what he will see while presenting, whathis main control device will be, what other users will see, how otherpresenters may interact by default, what the constraints are on a user'sability to change the stage and activate other widgets and assets, andwhat other connected devices and systems to integrate with.

Referring to FIG. 6B, in 3, Professor X begins his presentation usingvideo and a document loaded into a document viewing/editing widget.Presenter X's Tablet PC controller screen running an editor modeembodiment is depicted showing a full set of controls 7601 available forcontrolling/editing the information presentations display and changes ofstate. Other users' device's screens are depicted running either editormode or runtime mode embodiments, and will show a limited set ofcontrols 7602 for controlling/editing the information presentation,based on the constraints being set and managed by Presenter X and/orother presenters, using various embodiments of the system andapplication.

In 4, Professor X next chooses to display one or more poll questions tothe users and audience, using the polling widget. In this embodiment heis able to select a poll/survey question from a bank of existingquestions or he can create a new question. Additionally, a depiction ofwhat a user my see on their embodiment is included, wherein such user'sembodiment is displaying a poll response window (part of the pollingwidget). As with all embodiments, users can be either in the liveaudience or the remote audience, and can be using a desktop PC, amobile/smartphone, a tablet PC, a television, or any other computing oraudiovisual device.

Referring to FIG. 76C, in 5, Professor X monitors the poll's results.When he is satisfied that all or enough users have submitted theirresponses, he can close the poll and choose whether or not to displaythe results to the users.

In 6, Presenter X decides to do a hands-on demonstration for the usersusing the document camera. Presenter X uses the integrations widget toswitch the video feed to the connected document camera. Presenter Xclicks/selects the tools icon to access the integrations widget and ispresented with a menu of relevant options based on the state of theinformation presentation and other context. Presenter X is able toselect various routing options for this document camera video, includingadding a new video to the stage is this scenario he chooses to route thedocument camera to the screen in place of the current video on thestage.

Referring to FIG. 76D, in 7, an example is depicted of what may bedisplayed in a user's embodiment after Presenter X activates thedocument camera as described above. What is shown is the video on thestage of Presenter X doing a pen and paper based activity, it isimportant to note that any type of camera or specialized camera (e.g.,surgical camera, etc) may be used.

In 8, after Presenter X completes the hands on demonstrations using thedocument camera, he decides he will do a demonstration using screensharing, such that he may display his computer/tablet/device desktop andits running applications. Using his chosen device, Presenter X selectsto activate the screen sharing widget, and opts to use full screensharing over the whole stage, rather than replacing a particular widgetthat may be currently visible to the users on the stage of thecreation/playback application 120.

In 9, an example is depicted of what users, as well as Presenter X, maysee while Presenter X is using the screen sharing widget. Presenter Xsees a full set of controls 7601 over his desktop (which he can alsohide). Users will see limited set of controls based on constraints, andare able to see Presenter X's desktop.

FIG. 77 illustrates a single track embodiment of the creation playbackapplication 120 and various capabilities of the system in regard todiscussion beards. In this illustration, a discussion widget isdisplayed on the right with a slide show viewing/editing widget on theleft. In 1, a discussion board post that is rich text and contains animage is displayed. In 2, a new discussion board post is created by theuser. The document editing widget is visible and open within thediscussion board, such that the user can create and submit a rich textdiscussion post/message, which will be saved to the discussion widgetand system. In 3, it is shown how a discussion or post can be linked toa specific time in the video. In 3, the time in the video that thispost/link is related/tied to is 2:41:55. Clicking on the play buttonnext to the time (dark gray arrow) will play the informationpresentation starting from 2:42:55. In 4, the create link popup (part ofthe linking widget) is shown that a user can see if they attempt tocreate a link from any object or text in the discussion board widget orpost. In 5 a completed post is shown with a link (bright blue) madeusing the linking widget.

It is important to note that the sequences of screenshots illustrated inthis disclosure can represent: 1) an information presentation beingdelivered and controlled live and in real-time by an editor user andviewed live by one or more users; or 2) an information presentationwhich was predefined and is being played back by users at a later time;or 3) any combination thereof.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It will be apparent to persons skilled inthe relevant art(s) that various changes in form and detail can be madetherein without departing from the spirit and scope of the presentinvention. Thus, the present invention should not be limited by any ofthe above-described exemplary embodiments.

In addition, it should be understood that the figures described above,which highlight the functionality and advantages of the presentinvention, are presented for example purposes only. The architecture ofthe present invention is sufficiently flexible and configurable, suchthat it may he utilized in ways other than that shown in the figures.

In addition, it should be noted that the terms “a”, “an”, “the”, “said”,etc. in the specification, figures and claims signify “at least one”,“the at least one”, “said at least one”, etc.

Further, the purpose of the Abstract of the Disclosure is to enable theU.S. Patent and Trademark Office and the public generally, andespecially the scientists, engineers and practitioners in the art whoare not familiar with patent or legal terms or phraseology, to determinequickly from a cursory inspection the nature and essence of thetechnical disclosure of the application. The Abstract of the Disclosureis not intended to be limiting as to the scope of the present inventionin any way.

Finally, it is the applicant's intent that only claims that include theexpress language “means for” or “step for” be interpreted under 35U.S.C. 112, paragraph 6. Claims that do not expressly include the phrase“means for” or “step for” arc not to be interpreted less than 35 U.S.C.112, paragraph 6.

We claim:
 1. A system for creating at least one presentation,comprising: at least one computer; at least one application operable onthe at least one computer, the at least one application configured for:receiving a plurality of assets; and allowing at least one user todefine the at least one presentation over the time that the at least onepresentation is to be displayed, using at least two assets to besimultaneously presented for at least one selected point in time.
 2. Thesystem of claim 1, wherein the at least one application is furtherconfigured for: displaying the at least one presentation.
 3. The systemof claim 1, wherein the assets are interactive and/or non-interactive.4. The system of claim 3, wherein the interactive assets comprise atleast one widget.
 5. The system of claim 3, wherein the non-interactiveassets are real-time and/or non real-time.
 6. The system of claim 4,wherein the at least one widget is configured for: gathering user input;providing dialogue with at least one user; providing interactivitybetween at least one user and the at least one presentation; providinginteractivity between at least one user and at least one asset; orproviding interactivity between multiple users; or any combinationthereof.
 7. The system of claim 4, wherein the at least one widgetcomprises: at least one text chat widget, at least one audio/video chatwidget, at least one asset browser widget, at least one discussionwidget, at least one audio recording/playback widget, at least one videorecording/playback widget, at least one stream playback widget, at leastone audio/video editing widget, at least one document viewing widget, atleast one document editing widget, at least one linking widget, at leastone activity widget, at least one slide show editing/viewing widget, atleast one diagram editing/viewing widget, at least one drawing suitewidget, at least one whiteboard widget, at least one polling widget, atleast one survey widget, at least one assessment widget, at least onespreadsheet widget, at least one user list widget, at least one sourcecode viewing and editing widget, at least one tracking widget, at leastone glossary widget, at least one screen sharing and/or screen recordingwidget, at least one wiki widget, at least one bookmark widget, at leastone calculator widget, at least one presentation index widget, at leastone layers widget, at least one integrations widget, or at least onethird party platform interactive widget, or any combination thereof. 8.The system of claim 2, wherein the at least one application is furtherconfigured for: displaying the assets in at least one layoutconfiguration.
 9. The system of claim 2, wherein the displaying can bedone in automatic or manual mode.
 10. The system of claim 9, wherein theat least one user can determine which mode to use.
 11. The system ofclaim 1, wherein the at least one computer is: non-networked; in atelevision; connected with at least one intranet; connected with atleast one Internet; or in at least one mobile device; or any combinationthereof.
 12. The system of claim 11, wherein the at least one mobiledevice comprises: at least one laptop; at least one phone; at least onepersonal assistant device; at least one tablet computer; or anycombination thereof.
 13. The system of claim 2, wherein at least onewidget can be displayed on top of at least one other widget.
 14. Thesystem of claim 1, wherein the at least one application is furtherconfigured for allowing at least one user to define at least oneproposed layout of the at least one presentation.
 15. The system ofclaim 1, wherein the at least one application is further configured for:interacting directly and/or indirectly with at least one user; orinteracting between at least one user and at least one other user; orany combination thereof.
 16. The system of claim 1, wherein the at leastone application is further configured for: allowing at least one editorto define the at least one presentation by utilizing at least one trackfor defining the state of the at least one presentation and/or at leastone change of the at least one state of the at least one presentation.17. The system of claim 16, wherein the at least one track comprises atleast one asset track and/or at least one control track.
 18. The systemof claim 1, wherein the at least one application is further configuredfor predicting the at least one editor's intentions relating to definingthe at least one presentation by determining where the at least oneeditor clicks on at least one screen.
 19. The system of claim 18,wherein the at least one application is further configured forpresenting the at least one editor with at least one option for definingthe at least one presentation.
 20. The system of claim 1, wherein the atleast one application is further configured for defining constraintsrelated to at least one user's ability to modify the at least onepresentation.